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1.0  EXECUTIVE  SUMMARY 


A  panel  of  computer  scientists  was  tasked  to  define  a  ten-year 
research  and  development  program  for  RADC's  Requirements 
Engineering  Testbed.  The  purpose  of  this  report  Is  to  present  the 
panel's  findings  and  recommendations. 

1.1  General  Background  and  RADC's  Goals 

RADC's  charter  to  the  panel,  to  define  an  R&D  program,  was  based 
on  existing  RADC  goals  and  plans.  These  are  discussed  below. 

Incorrect  requ I rements  lead  to  the  development  of  systems  that 
cannot  be  fielded  because  they  do  not  address  all  critical  mission 
needs.  Within  the  Air  Force,  existing  requirements  tools  and 
methods  go  largely  unused  because  they  fall  short  of  what  Is 
needed  to  produce  correct  requirements  for  complex  systems  and 
software.  Even  among  software  contractors,  existing  tools  and 
methods  have  not  been  quickly  adopted.  Air  Force  users  have  great 
difficulty  in  recognizing  and  tracking  their  mission  concerns  in 
requirements  specifications,  because  they  tend  to  be  large, 
complex,  and  written  for  the  technical  community.  Thus,  Air  Force 
users  need  better  tools  and  methods. 

Therefore,  RADC  proposes  to  develop  a  Requirements  Engineering 
Testbec  (RET)  to  support  the  development  and  evaluation  of  new 
tools  and  methods.  Air  Force  users  would  use  the  RET  to  define 
and  exercise  their  requirements  for  planned  operational  systems. 

A  secondary  goal  of  RADC  is  to  encourage  use  of  the  new  tools  by 
Air  Force  users  and  industry;  the  RET  would  be  the  vehicle. 

In  the  near  term,  RADC  is  developing  prototyping  tools  that  will 
assist  Air  Force  users  in  requirements  understanding  and  in 
investigating  their  implications.  RADC  plans  to  have  these  tools 
hosted  in  the  RET  by  1988,  with  additional  requirements 
engineering  capabilities  added  by  1990. 

1.2  Panel.  Findings,  .and  Recommendations 

To  help  achieve  their  RET  goals,  RADC  identified  the  need  for  a 
ten-year  research  and  development  (R&D)  program  to  guide  them  In 
the  development  of  new  tools  and  techniques.  RADC  appointed  a 
panel  of  computer  scientists  from  academia  and  industry  to  define 
such  a  program. 

To  help  define  the  objectives  for  the  R&D  program,  the  panel 
created: 

(1)  a  model  of  the  requirements  engineering  process, 
capturing  the  panel's  understanding  of  that  process;  and 

(2)  two  scenarios  illustrating  the  process  model  in  terms  of 
RET  capabilities,  one  for  1990  and  the  other  for  1995. 


1 


The  node!  provided  a  basis  for  defining  the  R&D  program,:  the  R&D 
program  would  have  to  provide  tools  and  methods  to  support  process 
model  activities.  The  scenarios  provided  a  complementary  basis: 
the  R&D  program  would  have  to  provide  tools  and  methods  that  would 
work  together  In  the  way  envisioned  and  provide  the  Illustrated 
capab 1 1 Itles. 


The  panel  then  defined  the  R&D  program  as  the  strategy  to  bring 
all  this  about. 


The  panel  recommended  that  RADC  pursue: 


A  research  and  development  program  for  the  Requirements 
Engineering  Testbed  (RET)  consisting  of  two  tracks:  (1)  an 
Evolutionary  Track  for  developing  tools  and  methods  such  as  rapid 
prototyping  that  in  the  near  term  give  the  best  payoff  in  better 
requirements,  and  (2)  a  Formal  Language  Track  for  exploring  the 
h i gher  risk/payoff  implications  of  a  formal  requirements  language. 
Tre  risk  ir  the  Formal  Language  Track  is  that  one  must  be  able  to 
express  requirements  formally.  The  payoff  is  in  the  formal 
activities  that  can  be  automated.  Determining  requirements 
satisfaction  and  generation  of  scenarios  are  examples. 


The  panel's  1990  and  1995  RET  scenarios  respectively  illustrate 
capabilities  that  would  be  made  available  through  the  Evolutionary 
and  Formal  Language  Tracks.  The  panel  concluded  that  both,  tracks 
should  provide  the  RET  user  with  better  analysis  and  trade-off 
capabilities  than  are  current  I y  available. 


The  panel  further  recommended  a  testbed  Integration  plan  for  1990, 
that  would  lead  to  a  uniform  user  Interface  to  all  tools  and  a 
common  repository  for  all  requirements,  designs,  and  tool  data, 
and  which  incorporates  RADC's  currentl y-contracted  requirements 
tools.  This  recommendation  is  based  on  a  short-term  goal  of  a 
loose  coup! Ing  of  the  tools. 


A  review  of  this  report  was  conducted.  The  review  results  are 
presented  in  an  epilogue  (section  6.0).  In  general,  the  report 
was  found  to  be  techlncally  sound. 


The  R£D  program  consists  of  R&D  thrusts  In  these  areas:  (1) 
prototyping,  (2)  requirements  analysis,  (3)  tool  Integration  and 
evaluation,  and  (4)  a  formal  language  for  requirements  and 
specifications.  The  objective  of  the  Evolutionary  Track  Is  to 
provide  better  capabilities  and  more  automation  in  the  first  two 
areas  and  to  accomplish  and  support  the  third  area.  The  focus  of 
the  Formal  Language  Track  Is  the  last  area.  Below,  we  summarize 
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each  area,  Identifying  capabilities  to  be  developed  by  1590  and 
1995,  anc  how  these  relate  to  P.ET  goals. 

Prototyping 

In  prototyping,  the  1990  goal  is  to  develop  capabilities  In:  (1) 
prototyping  system  Interfaces  and  functions,  (2)  developing 
scenarios  to  drive  the  prototypes  In  experiments,  and  (3) 
collecting  and  analyzing  results.  The  1995  goal  Is  to  extend 
capabilities  for  analyzing  prototyping  results. 

Relationship  to  RET  goals:  End  users  are  general ly  able  to 
analyze  their  mission  concerns  better  In  operational  settings  than 
by  reading  lengthy,  textual  requ I rements  specifications.  Thus 
prototyping  will  play  an  Important  role  in  discovering  and 
understanding  critical  needs,  and  therefore  In  capturing  these 
needs  in  the  requirements.  Prototyping  will  also  provide  a 
feasibility  check,  and  when  later  supplemented  with  performance 
analysis,  will  serve  as  a  basis  for  sensitivity  analysis  on 
requ i rements . 


Requirements  Analysis 

Ir  requirements  analysis,  the  1590-1595  goals  are  to  be  able  to 
analyze  requirements  to  investigate  assumptions,  decisions, 
implications,  and  requ i rements  quality:  (1)  expected  cost  and 
risk  of  developing  the  envisioned  system,  (2)  expected  cost, 
performance,  and  reliability  of  operating  the  envisioned  system, 
(3)  the  rationale  for  a  particular  mission  need  or  trade-off 
cecisicn,  (4)  requirements  related  to  specific  needs  and 
expectations  of  a  particular  end  user,  and  (5)  which  requirements 
are  untesteble,  contrad ictory,  or  redundant.  The  key  to  achieving 
these  goals  will  be  to  represent  requirements  more  precisely, 
strongly  couple  the  analysis  to  requirements  updates,  and  identify 
relevant  metrics.  Another  goal  is  to  provide  a  methodology  to 
guice  the  user  in  the  use  of  these  analysis  capabilities. 

Relationship  to  RET  goals:  Requirements  correctness  and  quality 
will  improve.  Air  Force  users  will  use  the  analysis  tools  to 
Investigate  their  concerns  In  the  requirements  Including  the 
rationale  and  Implications  of  decisions  that  were  made. 
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Tool.  Integration  and  Evaluation 

In  the  integration  and  evaluation  of  tools,  the  1990  goal  Is  to 
provide  state-of-the-art  database  and  human  Interface  capabilities 
as  a  basis  for  an  Integrated  testbed  of  tools  and  for  monitoring 
tool  performance.  The  1995  goal  Is  to  Integrate  new  prototyping, 
requirements  analysis,  and  formal  language  capabilities  Into  the 
instrumented  RET  to  facilitate  their  evaluation. 

Relationship  to  RET  goals:  A  common  database  implies  tools  will 
be  able  to  share  data.  A  common  user  interface  Implies  user 
learning  time  will  be  reduced.  Both  imply  the  user  will  be  able 


to  nove  easily  from  one  tool  4o  another.  Development  efforts  will 
be  reduced  -  new  tools  can  use  the  same  data  management  facilities 
and  utilize  the  same  human  Interface  mechanisms  for  user 
communication.  An  Instrumented  and  Integrated  testbed  is  a  basis 
for  evaluating  relative  tool  effectiveness. 

Formal  Language 

In  the  formal  language  area,  the  1990  goal  Is  to  provide:  (1)  a 
common  formal  language  for  requirements  and  specifications  and  (2) 
tools  that  automatically  compare  requirements  and  specifications 
statements.  The  1995  goal  Is  to  provide:  (1)  the  capability  to 
generate  and  Interpret  scenarios  and  (2)  extensions  to  the 
language  that  facilitate  Incremental  modifications  and  multiple 
levels  of  abstraction. 

Relationship  to  RET  goals:  Requirements  statements  will  be 
precise  and  mach i ne- I nterpretab I e,  leading  to  increased  automation 
and  support  for  prototyping,  requirements  analysis,  and 
development  activities.  These  implications  are  long-term 
consequences  of  the  formal  language  approach. 


The  purpose  cf  this  section  is  to  establish  a  context  for 
understanding  the  panel's  recommendations.  The  panel's 
recommendations  are  presented  in  sections  3  and  4. 

In  summary,  the  context  is  this:  (1)  the  Air  Force  needs  better 
requirements  tools  and  techniques,  (2)  RAOC's  approach  to  meeting 
this  need  is  to  create  a  testbed  for  development  and  evaluation  of 
new  requirements  tools  and  techniques,  (3)  RADC  convenes  a  panel 
of  computer  scientists  to  refine  the  testbed  plan  and  to  recommend 
a  testbed  research  and  development  program,  and  (4)  the  panel 
makes  recommendations  based  on  the  current  state  of  the 
requirements  process:  available  tools,  characteristics  of 
requ I rements,  and  methods  of  checking  requirements. 

Before  dealing  with  the  context  In  detail,  we  define  seme  terms. 

.Req.ui.rer.ents  Terminology 

Requirements  are  precise  statements  of  need  which  characterize  a 
reeded  system  in  terms  of  its  external  characteristics  (especially 
interfaces  and  functionality  visible  to  the  user)  and  constraints 
(e.g.  or  execution  performance  and  reliability,  and  on  development 
cost  and  tire). 

Requirements  engineering  is  the  systematic  application  of  tools 
and  techniaues  to  guide  and  control  the  emergence  of  the 
requirements  product.  It  is  an  iterative  process  of  analysis, 
evaluation,  and  refinement.  Its  inputs  are  mission-related  ideas 
end  problems  expressed  by  mission  specialists  and  their 
representatives.  Its  output  is  the  set  of  requirements. 

Air  Force  Requirements  Problem 

As  part  of  an  RADC-sponsored  effort,  an  informal  survey  of  Air 
Force  users  (specifically,  mission  users  and  acquisition 
engineers)  and  contractors  was  conducted.  The  survey  consisted  of 
25  interviews  with  both  Air  Force  users,  other  Department  of 
Defense  users,  and  contractors.  The  survey  Identified  three  major 
problem  areas: 

(1)  Requirements  specifications  were  written  for  procurers 
(acquisition  engineers)  and  their  technical  staffs.  Thus 
mission  users  found  them  to  be  too  technical  and  felt  "shut 
out”. 

(2)  Mission  users  and  contractors  found  It  difficult  to 
relate  A-spec  to  B-spec  (i.e.  high-level  system 
specification  to  software  requirements)  because  of  the 
significant  "gulf  between  them. 


(3)  Contractors  and  mission  users  complained  that 
traceability  cou I  a  only  be  demonstrated  manually,  making  it 
hard  to  assess  requirements  coverage. 

In  summary,  Air  Force  users  found  It  difficult  to  preserve  a 
recognizable  representation  of  mission  user  concerns  in  the 
derived  specifications. 

The  survey  also  found  that  most  mission  users  who  could  have  used 
a  requirements  method/tool  (e.g.  PSL/PSA)  never  did,  and  not  one 
of  those  who  did  considered  the  use  to  be  a  success.  So 
requirements  engineering  methods  and  tools  are  underutilized  by 
Air  Force  users.  They  are  based  on  technologies  such  as:  formal 
specification  languages,  consistency  checkers,  and  report 
generators.  Those  methods  and  tools  that  are  not  difficult  to 
master  do  little  more  than  check  how  well  things  fit  together 
syntactically.  Instead  of  stating  his  needs  and  making  decisions 
at  the  mission  level,  the  user  is  forced  to  work  at  a  design  level 
or  use  the  English  language.  The  cone  I  us  ion:  existing  methods 
and  tools  match  the  needs  and  abilities  of  system  developers 
instead  of  Air  Force  users. 
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To  deal  with  the  Air  Force  requirements  problem,  RADC’s  long-term 
goal  is  to  develop  and  evaluate  new  tools  and  methods  that  will 
enable  Air  Force  users  to  bring  new  technologies  to  bear  on  their 
requirements  engineering  problems. 

Eut  these  new  tools  and  methods  cannot  be  developed  in  isolation. 
They  need  to  be  evaluated  on  realistic  problems  and  placed  into  an 
environment  that  supports  their  use  by  Air  Force  users. 

Therefore  RADC  proposes  the  creation  of  a  facility  which  promotes 
experimentation,  called  the  Requirements  Engineering  Testbed 
(RET),  and  which  hosts  the  new  methods  and  tools.  Air  Force  users 
will  use  the  RET  to  define,  analyze,  and  exercise  requirements  of 
planned  operational  systems.  This  will  have  two  benefits.  First, 
new  tools  and  methods  will  have  early  exposure  on  realistic 
problems,  encouraging  their  use  by  Air  Force  users  and  Industry. 
Second,  evaluation  of  the  new  tools  and  methods  In  terms  of 
requirements  quality  and  productivity  will  provide  Insight  for  the 
next  generation  of  tools. 


We  next  characterize  the  users  of  the  RET.  This  is  followed  by  a 
discussion  on  hosting  RADC’s  currently-contracted  prototyping 
tcols  in  the  RET  in  early  1988,  and  a  subsequent  integration. 


The  RET  must  support  the  requirements  engineering  activities  of 
three  user  classes  in  the  evaluation  of  new  tools  and  methods: 

Air  Force  mission  users,  Air  Force  acquisition  engineers,  and 
contractor  software  developers.  Figure  2.2-1  defines  their  Jobs 
and  character izes  their  present  and  projected  future  familiarity 
with  computers  and  programming. 

The  RET  must  concentrate  on  supporting  the  mission  user  and 
acquisition  engineer  In  keeping  with  RADC's  mission  and  because 
support  for  these  classes  has  been  neglected  by  the  current 
generation  of  tools. 

These  two  users  have  different  requirements  concerns  and 
interests.  Mission  users  are  interested  in:  the  data  (messages) 
which  must  be  processed,  the  user  interface,  performance  (e.g. 
workstation  ergonomics  and  work  flow),  and  scenarios.  Acquisition 
engineers  are  Interested  in:  functional  characteristics  of  the 
target  system.  Its  software  requirements  (database  management 
systen,  utilities),  testing,  and  performance. 

In  order  tc  better  understand  the  context  of  the  RET,  the  panel 
developed  a  rore  detailed  character izat ion  of  the  three  user 
classes.  Appendix  A  presents  the  panel's  characterization. 

Near-Terr  RET  Prototyping  Tools. 

RACC  currently  has  three  different  prototyping  tools  under 
contract  to  be  developed.  In  the  near  term  (early  1988),  these 
+hree  tools  will  be  hosted  by  the  RET  and  will  provide  its  initial 
requirements  capabilities: 

(1)  The  Analyst,  an  existing  requirements  tool  currently 
being  enhanced  by  Systems  Designers  under  subcontract  to 
Imperial  College,  will  assist  in  the  creation  of  a 
semiformal  problem  domain  description.  The  tool  also  aids 
documentation  of  performance  and  reliability  stress  points. 

A  prototyping  capability  will  estimate  performance  based  on 
estimated  work  loads. 


(2)  The  VHLL  Prototyping  tools,  to  be  developed  by 
International  Software  Systems,  will  assist  In  the  creation 
of  functional  prototypes.  A  VHLL  (very  high  level  language) 
Is  used  to  specify  the  desired  functionality  of  the  target 
system.  The  resulting  description  Is  executable  and  can  be 
exercised  with  different  Inputs  dynamically. 

(3)  The  Rapid  Prototyping  System,  to  be  developed  by  Martin 
Marietta,  takes  a  node  I -based  approach  to  prototyping; 
aiding  the  production  of  operator  models,  processor  models, 
scenario-driven  functional  models,  and  communications 
models.  Within  these  models,  system  performance  can  be 
estimated  based  on  estimated  work  loads  and  target  system 


configuration.  There  Is  also  a  capability  provided  for 
rapidly  designing  interfaces  with  dynamic  graphics. 

RADC  plans  an  Integration  effort,  which  would  start  soon  after 
delivery  of  these  tools,  and  would  result  in  a  1990  testbed  of 
integrated  tools. 


2.3  Requirements  Panel  Objectives 

To  properly  plan  the  Introduction  of  new  methods  and  tools  into 
the  RET  and  to  help  in  their  evaluation,  a  long-term  research  and 
development  (R&D)  program  needed  to  be  defined.  To  assist  in  the 
definition  of  the  R&D  program,  a  panel  of  computer  scientists  from 
academia  and  Industry  was  appointed  and  convened  four  times  during 
July  1985  to  February  1986. 

The  members  of  the  panel  were: 

Robert  Balzer  -  Information  Sc i .  Inst.,  Marina  del  Rey,  CA 

*  ’-’ichael  Konrad  -  International  Software  Systems,  Austin,  TX 
C.  V.  Ramamoorthy  -  University  of  California  at  Berkeley,  CA 
Winston  Royce  -  Lockheed  Missiles  &  Space,  Austin,  TX 

*  William  Rzepka  -  Rome  Air  Development  Center,  Rome,  NY 
Steve  Sherman  -  Lockheed  Missiles  &  Space,  Austin,  TX 
Leon  Stuck i  -  Future  Tech,  Auburn,  WA 

*  Terry  Welch  -  International  Software  Systems,  Austin,  TX 
Raymond  Yeh  -  International  Software  Systems,  Austin,  TX 


*  Rzepka  was  panel  sponsor.  Welch  was  panel  chairperson. 

Konrad  was  panel  report  editor. 

Fei  Fsia  of  the  University  of  Texas  at  Arlington,  Texas, 
contributed  to  the  panel's  efforts. 

As  stated  earlier,  the  primary  objective  of  the  panel  was  to 
define  a  ten-year  R&D  program. 

A  secondary  panel  objective  was  to  Identify  what  additional  effort 
was  required  to  make  a  full  range  of  requirements  engineering 
capabilities  accessible  to  the  RET  user  by  1990,  based  on  the 
currently  contracted  tools. 

The  remainder  of  this  report  presents  the  panel's  findings  and 
recommendations,  except  section  6.0,  which  presents  results  of  a 
review  of  the  panel's  findings  and  recommendations. 


2. a  Current  State  of  the  Pecuirenents  Process 

In  this  section  we  characterize  the  current  state  of  the 
requirements  process  as  viewed  by  the  panel. 
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The  current  state  of  the  requirements  process  Is  that  it  is  almost 
entirely  a  manual  process  whose  success  depends  on  the  insight  of 
the  analyst  doing  the  work.  There  are  no  general ly  accepted 
methodologies,  criteria  for  quality,  or  notations. 

Requirements  are  almost  entirely  English  text,  so  they  are  subject 
to  differing  interpretat ions  by  mission  experts  and  by  system 
developers,  namely  the  two  audiences  that  the  requirements 
statements  should  bring  together. 

Evaluation  of  Existing  Reau Irements  Techniques 


Existing  requirements  techniques,  such  as  SREM  and  PSL/PSA, 
provide  a  methodology  for  requirements  analysis.  The  methodology 
is  supported  by  tools:  (1)  mach Ine-processab I e  languages  for 
stating  the  requ  I  rerrents,  (2)  static  analysts  tools  for 
determining  consistency  and  completeness  of  the  statements,  (3) 
simulation  tools  for  assessing  feasibility,  and  (4)  report 
generators.  As  an  example  we  consider  SREM  (ciscussicn  is  based 
on  QStonej  and  CSchef f erj) : 

characteristics  can  be  expressed,  e.g.  the  system  interface 
in  terms  of  message  flow  and  content,  and  system  software 
functions  in  terms  of  inputs,  outputs,  and  proces^'ng 
cependencies.  Certain  characteristics  are  very  difficult  to 
express  in  the  language  and  so  analysts  must  rely  on  textual 
representations:  real-time  and  near-real-time 
characteristics,  parallel  end  distributed  processing,  nan- 
machire  interfacing. 

(2)  Analysis  tool s.  Analysis  is  performed  in  batch  and  results 
are  presented  in  generated  reports.  Two  kinds  of  analysis 
are  performed:  (1)  data  analysis  consisting  of  syntactic 
checks  on  language  statements  (i.e.  detection  of 
syntactically  missing  or  misused  elements),  and  (2)  data 
flow  analysis  consisting  of  checks  that  consuming  elements 
process  exactly  those  elements  named  In  their  Input.  The 
SREM  evaluators  found  that  most  requirements  errors  (80$  or 
higher)  are  caught  by  the  analyst  while  using  the 
methodology,  not  by  the  analysis  tools. 

(3)  Simulation  tools.  Simulations  are  partly  "generated",  but 
analysis  of  results  Is  manual.  The  simulations  aid 
verification  of  (1)  system  Interfaces  and  processing 
relationships,  and  (2)  analytic  feasibility  (is  there  a 
testable  design  that  satisfies  the  accuracy  requirements). 

In  practice,  too  much  labor  Is  Involved  to  justify  more  than 
verification  of  system  Interfaces. 

requirements  phase  and  early  In  the  design  phase.  Utility 
of  SREM  lies  In  defining,  correcting,  and  analyzing  the 
software-aspects  of  a  system,  I.e.  after  allocation  of 
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requirements  to  functional  software  components.  SREM 
provides  little  support  in  the  "conceptual  phase": 
expressing  needs,  assessing  feasibility,  trade-off  studies. 

Recently  ^Alford  1985],  SREM  is  being  extended  to  address 
concerns  that  arise  earlier  In  the  system  requirements  phase 
and  also  detailed  Issues  that  arise  In  later  design  and 
testing  phases. 

In  summary,  existing  techniques  provide  a  discipline  for  defining 
requirements  and  analyzing  them,  but  are  best  at  addressing  late 
"requ 1 rements  phase"  concerns.  They  require  a  largely  manual 
execution  of  the  requirements  process.  It  Is  not  yet  evident  that 
the  labor  Involved  In  their  use  is  Justified  by  the  improved 
results  obtained. 

Problems  In.  Comrun  icat  jpg  and  Analyzing  Retirements 

The  panel  members  shared  these  basic  assumptions  about 
requirements  which  guided  their  thinking: 

1.  Formal  Descriptions.  Requirements  must  be  stated  with 
increased  precision.  A  formal  basis  for  requirements  will 
lead  to  machine  interpretation,  an  important  step  in 
achieving  Increased  productivity  in  the  design  and 
verification  of  software  systems. 

2.  Domain  Information.  Requ irements  are  often  ambiguous  with 
respect  to  assumptions  and  decisions  based  on  the  "common 
sense"  cf  such  things  as  the  laws  of  physics,  psychology  of 
people,  politics  cf  organ i zat ions,  software  algorithms, 
application-specific  technology  (e.g.  how  does  a  radar 
work),  etc.  Mechanized  interpretation  of  requirements 
necessitates  capturing  and  interpreting  this  context 
information. 

3.  Prototyping.  Communication  of  requirements  statements  to 
mission  experts  often  requires  building  a  functional 
prototype  to  animate  those  requirements.  This  is  because  of 
the  poor  communication  capability  of  requirements  statements 
and  because  many  individuals  are  better  able  to  analyze 
their  mission  needs  when  they  see  operational  results. 

4.  Development  Facilities.  Requirements  development,  like  any 
other  manual  design  activity,  should  be  supported  by  a 
workstation  environment  which  provides  rapid  interaction  via 
graphics  tightly  coupled  to  an  object-oriented  database. 

This  type  of  facility  will  be  commonplace  in  five  years  and 
provides  important  capabilities  for  humans  deal  ■'g  with 
complexity  in  large  system  designs. 
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Below  we  describe  four  methods  In  use  today  to  assess  the  validity 
of  a  set  of  requirements.  Each  of  these  methods  is  presently  very 
labor  intensive,  but  might  be  partially  mechanized  If  requirements 
statements  were  more  mach ine-man ipu latab I e. 

1.  Consistency  Checks.  Requirements  statements  as  a  set  can 
be  checked  for  contradictions  and  completeness.  Current 
techniques,  such  as  PSL/PSA  and  SREM,  provide  modest  forms 
(e.g.  dataflow)  of  checking.  Otherwise,  the  checks  are 
manual . 

2.  Scenarios.  Scenarios  developed  to  1 1  lustrate  typical  or 
stressful  system  usage  serve  to  provide  a  cross  check  on 
requirements.  The  development  of  precise  scenarios  is  very 
time  consuming,  however,  and  checking  them  against 
requirements  statements  is  not  easy. 

3.  Prototyp i ng.  The  development  of  a  prototype  provides  a 
check  on  the  imp  I ementsb  i  I ity  of  the  requirements,  as  well 
as  providing  a  means  of  eliciting  user  feedback.  To 
minimize  prototype  development  effort,  one  generally 
restricts  the  requirements  subset  tested  (e.g.  omitting 
error  handling  and  performance)  and  reduces  prototype 
development  "overhead"  (e.g.  documentation). 

4.  Cost  Estimates.  Major  factors  influencing  any  system  design 
are  the  costs  and  risks  in  development  and  operation.  Thus 
these  factors  must  be  considered  when  evaluating 

requ i renents.  Present  mechanisms  for  estimating  cost  and 
risk  are  notably  intuitive  and  often  ineffective. 

Each  of  the  above  four  areas  warrants  significant  research  on 
possible  automation  and/or  mechanical  support  for  human  execution. 
Success  in  this  work,  however,  certainly  depends  on  Increased 
precision  and  machine  interpretation  of  requirements,  so  work  on 
formalizing  requirements  descriptions  Is  a  cornerstone  of 
requ i rements  research . 


THE  REQUIREMENTS  ENGINEERING  PRXESS  MODEL. 


In  an  effort  to  come  to  a  common  understanding  of  the  requirements 
engineering  process,  especially  terminology,  and  to  partially 
capture  that  understanding,  the  panel  created  a  model  of  the 
process.  This  section  presents  that  model  and  Illustrates  It  with 
scenarios  indicating  RET  capabilities  In  1990  and  1995. 

The  appendices  contain  more  discussion  and  further  details  on  the 
process  model.  Appendix  B  relates  process  model  terminology  to 
the  current  Air  Force  requirements  engineering  practice.  Appendix 
C  gives  detailed  character tzat Ions  of  process  model  Information 
types  and  activities.  Appendix  F  discusses  philosophical  problems 
Inherent  in  requirements  engineering  that  can  have  no  solution  In 
terms  of  a  "better  tool". 

The  Model 

The  model  is  portrayed  in  figure  3.1-1.  The  figure  shows 
Imormation  depicted  as  boxes;  and  activities  depicted  as  circles, 
cvals,  and  rounded-corner  boxes. 

The  model  indicates  the  dependencies  and  sequencing  between 
activities,  but  is  not  meant  to  favor  a  particular  methodology. 

loigrr.ft.ii.cn.,,  lyp.ep 

The  model  identifies  three  major  types  of  information;  goals, 
requirements,  and  solution  architectures. 

Goa  I s  are  expressions  of  objectives  and  needs,  generally  mission- 
related,  and  not  necessarily  feasible  or  consistent  with  each 
ether.  Mission  users  are  the  primary  source. 

Requ i rements  are  a  consistent  subset  of  the  goals  which  can  be 
feasibly  realized  within  the  available  resources  (especially  time, 
money,  and  expertise). 

A  Solution  architecture  Is  a  model  of  the  target  system  as  a 
composition  of  parts  that  satisfy  the  requirements.  The  more 
common  term  Is  spec  1 f Icat Ion,  but  the  panel  preferred  solution 
architecture,  as  specification  has  been  used  to  mean  different 
th I ngs. 
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We  take  a  top-down  walk  through  the  requirements  engineering 
process  model.  Identifying  the  objects  and  activities  depicted  in 
the  figure.  For  the  walk-through,  we  assume  a  requirements 
engineer  is  required  to  produce  a  set  of  requirements  for  a  system 
called  the  "target  system". 


The  target  system  must  support  the  different  roles  of  its  users 
and  administrators.  Thus  there  will  be  different  expectations,  or 
"viewpoints"  of  what  the  system  should  do,  of  how  well  it  should 
be  done,  and  within  what  cost.  In  the  model  these  undocumented 
expectations  of  target  system  functions,  performance,  and  cost  are 
represented  by  Wish  Lists.  There  is  one  wish  list  per  role  or 
v i ewpo i nt . 


The  requirements  engineer  is  limited  in  the  time  he  can  expend  in 
the  creation  cf  target  system  requirements.  Thus  he  needs  to 
prioritize  his  objectives.  These  limitations  and  objectives 
should  both  be  documented.  In  the  model  they  are  represented  by 


Through  interviews  with  target  system  users/administrators  and 
through  references  to  documentation  of  similar,  existing  systems 
and  their  environments,  the  requirements  engineer  collects  and 
organizes  information  on  the  operational  context  of  the  target 
system.  The  resulting  Information  forms  a  Domain  Model  of  the 
environment  of  the  target  system,  providing  the  terminology  and 
context  through  which  wishes  can  then  be  expressed,  forming  Goals. 
Goals  represent  the  initial  attempt  at  documenting  a  system's 
desired  attainments.  For  each  viewpoint,  there  will  be  one  set  of 
goals,  and  they  should  be  consistent  and  complete  within  that 
v i ewpoi nt. 


Goals  are  often  inconsistent  across  viewpoints  or  clearly 
Infeasible.  Such  difficulties  must  be  resolved  by  the 
requirements  engineer  through  further  user  Interviews.  The 
revised  goals  are  then  merged  into  a  preliminary  set  of 
Requirements  for  the  target  system. 


Through  his  interviews  with  users,  the  requirements  engineer 
Identifies  and  documents  Scenar los  that  Illustrate  typical  target 
system  behavior  and/or  desired  responses  to  stressful  input. 
Scenario  construction  and  analysis  may  aid  stating  the 
nonfunctional  requirements,  in  particular,  performance  and 
re  I  lab  1 1 ity. 


During  the  creation  of  goals  and  requirements,  their  consistency 
and  completeness  Is  checked.  This  Is  called  Static  Analysis  In 
the  process  model.  To  determine  requirements  coverage,  the 
requirements  engineer  might  perform  a  walk-through,  analyzing  the 
dataflows  and/or  stimuli/responses  through  the  various  viewpoints. 
This  is  called  Dynamic  Analysis  In  the  process  model. 


15 
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At  this  point,  the  requ i rerents  engineer  might  construct  a 
Solution  Architecture  to  gain  better  insight  into:  target  system 
interfaces,  functions,  performance  and  reliability,  and  implied 
development  cost  and  risk. 


The  requirements  engineer  creates  a  solution  architecture  by 
specifying  how  the  target  system  is  composed  of  parts  (e.g. 
objects,  functions)  and  how  those  parts  use  resources  (e.g. 
people,  software,  hardware).  To  aid  specification  of  resources, 
the  requirements  engineer  can  make  reference  to  existing  resource 
model s. 


From  the  solution  architecture,  the  requirements  engineer  can 
specify  a  prototype.  He  executes  the  prototype  against  canned  or 
user-control  led  scenarios,  eliciting  user  comments  on  what  should 
be  changed.  Al I  of  these  activities  are  covered  by  the  Rap i d 
Prctctvc I ng  bubble  in  the  process  model. 


Also,  the  requirements  engineer  can  do  Analysis  directly  on  the 
solution  arch itecture.  By  combining  botn  rapid  prototyping  and 
analysis  activities  and  iterating,  the  requirements  engineer  can 
do  sensitivity  analyses. 


As  a  result  of  the  insights  gained  through  analysis  and  rapid 
prototyping,  the  requirements  engineer  determines  what  revisions 
should  be  made  and  makes  them.  This  activity  is  called 
Requirements  .Evaluation  L  Refornu !  at  ion  in  the  process  model. 


Tf-e^e  ray  be  several  iterations  of  prototype,  analyze,  evaluate 


and  reformulate.  The  resulting  requirements  and  solution 


architecture  Is  called  the  F i na I  P( 
Arch itecture  in  the  process  model. 


For  the  purpose  of  explaining  the  panel's  vision  of  near-term  and 
long-term  RET  capabilities,  the  panel  developed  two  scenarios,  one 
for  1990  and  one  for  1995.  Thus  these  scenarios  Identify  "where 
to  go";  the  RET  R&D  program,  detailed  In  section  4,  Illustrates 
the  panel's  strategy  of  how  to  get  there. 

We  make  free  use,  below,  of  process  model  terminology  and 
references  to  currently-contracted  tools. 

Scenario  Context: _ 19.9.0/1995  Objectives  for  the  RET 


The  panel  recommends  that  a  primary  1990  RET  objective  be  to 
provide  the  RET  user  with  a  wide  range  of  requirements  engineering 
capabilities  based  or,  three  integrated  tools:  the  Analyst,  the 
Rapid  Prototyping  System,  and  the  VHLL  System  Prototyping  tools. 

This  objective  is  the  context  for  the  1990  scenario,  whose  purpose 
is  to  illustrate  these  Evolutionary  Track  capabilities. 

Capab i I i t i es  from  the  ether  track  will  not  yet  be  general  I y 
avai I ab I e. 

Similarly,  a  primary  objective  for  the  1995  RET  is  to  provide  the 
RET  user  with  a  wide  range  of  capabilities  based  on  a  single 
formal  language  fer  expression  of  requ I  rerents,  solution 
architectures,  and  goals.  This  objective  Is  the  context  for  the 
1995  scenario,  which  thus  emphasizes  capabilities  developed  under 
the  Formal  Language  Track. 

Sc.erar.ic  St.er.t_i jig,  feint 

Both  scenarios  consider  the  same  nontrivial  application,  a  patient 
monitoring  system  (PMS),  and  address  essentially  the  same  trade¬ 
off  problem:  defining  requirements  for  a  PMS  of  low  operational 
cost  that  meets  the  need  for  Immediate  nurse  or  doctor 
notification  of  a  patient  problem.  Thus  both  scenarios  can  be 
compared  to  contrast  what  we  envision  can  be  done  in  1990  and 
1995. 

The  two  scenarios  have  the  same  starting  point:  the  requirements 
engineer  visits  the  RET  with  the  Intent  of  using  RET  tools  and 
techniques  to  define  PMS  requirements.  As  the  RET  is  a  facility 
for  tool  effectiveness  experiments,  careful  tracking  of  RET  user 
objectives  and  RET  resources  (e.g.  tools  and  RET  time)  Is 
necessary.  Toward  this  end,  the  "Engineering  Context  Description" 
documents  the  objectives  of  the  RET  exercise  and  how  much  effort 
can  be  spent.  For  both  1990  and  1995,  It  Is  expected  to  be 
maintained  as  mostly  unstructured  text  in  the  RET  database. 


Figure  3.3-1  Illustrates  the  Engineering  Context  Description 
considered  as  context  to  both  the  1990  and  1995  scenarios.  In 
both  scenarios,  the  objective  of  the  RET  exercise  Is  to  gain 
Insight  Into  the  trade-off  between  two  competing  wishes.  For  the 
1990  scenario,  a  third  wish  was  needed  to  motivate  the 
Illustration  of  the  currently  contracted  tools.  It  must  be 
Ignored  when  considering  the  1995  scenario. 

Where  do  these  wishes  come  from?  In  a  typical  PMS,  patients  are 
monitored  and  If  anything  goes  wrong  a  nurse  or  doctor  Is  alerted. 
So  the  three  obvious  PMS  users  are  the  doctor,  the  nurse,  and  the 
patient.  Less  obvious  Is  the  hospital  administrator  who  must  show 
a  profit.  In  both  scenarios.  It  Is  assumed  that  interviews  and 
relevant  documentation  have  revealed  these  needs:  (1)  doctors 
need  Immediate  notification  if  there  is  a  sericus  problem  (wish 
1),  (2)  hospital  administrator  needs  to  keep  operational  costs 
down  (wish  2),  and  (3)  (only  for  the  1990  scenario)  nurses  need 
help  tracking  patients'  care.  Typically,  investigation  will 
reveal  many  needs  forming  "Wish  Lists",  but  we  will  only  deal  with 
these  three. 


.  -  < 

■  *■  *  -  *  ■.*  O  *  '  i  ’  » * .  *  ^ 

.  *r.  - 

w-.; 

» 

LdJ 

V.  x  .  * 

VikJ!  k.  a  ^  am  afu,  A;  ir. 

I 


The  1990  RET  will  faci I ita+e: 

Partially  formal  characterizations  of: 

The  problem  domain 

Functional  requirements  and  some  nonfunctional  requirements 

Displays,  Interface  protocols 

Scenarios 

Solution  architectures 
Data  structures  (*) 

Support  for: 

Organizing  the  problem  domain 
A  functional  description  of  the  system 
Building  executable  models 
Building  performance  models  and  scenarios 
Exercising  prototypes  against  scenarios  (*) 

The  scenario  illustrates  this  RET  usage  paradign: 

Establish  RET  exercise  objectives 

Create  a  "domain  model"  documenting  user  roles  and  activities 
This  domain  model  will  expand  to  include  solution  resources 
Express  goals 

Goals  are  expressed  as  attributes  or  annotations  to  the 
domain  model 

Evaluate  for  feasibility  and  trade-offs: 

Explore  different  solution  approaches  end  then  revise 
Incorporate  analyses  and  prototyping  results  (*) 

Document  results  of  RET  exercise 

Traceability  of  goals  to  requ i renents  to  solutions  (*) 

*  (not  illustrated  in  1990  scenario) 

The  figures  for  this  scenario  illustrate  notations  that  are 
logically  equivalent  to  the  graphics/text  notations  that  will  be 
available  in  1990  RET  tools. 

Recalling  figure  3.3-1,  we  begin  the  1990  scenario.  We  refer  to 
the  hypothetical  RET  user  as  a  "requirements  engineer". 


Applying  the  CORE  method,  which  is  supported  by  the  Analyst  tool, 
the  requirements  engineer  first  Identifies  the  major  PMS 
viewpoints  (I.e.  the  functional  roles  with  which  PMS  Interacts). 
They  might  be  Illustrated  graphically  as  shown  In  figure  3. 3. 1-1. 
The  requirements  engineer  will  Interview  viewpoint  representatives 
(e.g.  the  head  nurse),  documenting  ail  relevant  Interactions  and 
w  i  shes. 


From  such  interviews,  the  requ irements  engineer  constructs  a 
domain  model  of  the  operational  environment  of  the  target  system. 

F igure  3.3. 1-2  illustrates  a  portion  of  such  a  model  (underl ined 
words  are  keywords).  The  domain  model  would  also  represent 
typical  "transactions”,  e.g.  "notifying  doctor  on  unsafe 
condition",  as  Illustrated  In  f Igure  3.3. 1-3. 

Figure  3.3. 1-2  Is  not  complete,  among  the  activities  not  shown 
here  but  referenced  later:  the  patient  can  request  a  nurse 
(activate  a  nurse  alarm)  and  PMS  will  pass  this  patient  request  to 
a  nurse. 


The  domain  model  (figures  3.3. 1-2  -  3.3. 1-3)  provides  much  context 
necessary  for  expressing  and  Interpreting  Goals.  Wishes 
constraining  operational  parameters  such  as  time,  reliability,  and 
operational  cost  will  often  be  expressed  In  1990  as  formal 
attributes  or  informal  annotations  to  transactions  (whether  formal 
or  informal  Is  not  yet  clear).  Two  of  our  figure  3.3-1  wishes  can 
be  sc  expressed:  the  doctor's  performance  and  reliability  wish 


and  the  hospital  admin Istrator *s  cost 
likely  be  expressed  in  informal  text. 


wish.  The  nurse's  wish 
See  f igure  3.3. 1-4. 


wil  I 


The  requirement  engineer's  objective  is  to  analyze  the  trade-cff 
between  these  performance  and  cost  wishes.  To  dc  this,  the 
reau i renents  engineer  will  treat  the  goals  as  preliminary 
requirements  and  explore  solutions. 

At  this  point  in  our  scenario,  we  assume  the  requ i rements  engineer 
interrupts  application  of  the  CORE  method  to  better  understand  the 
solution  implications.  The  knowledge  accrued  will  help  him  refine 
and  revise  the  goals  into  requirements. 


The  requirements  engineer  needs  to  consider  how  to  build  a  system 
that  will  satisfy  the  Requirements.  He  decides  on  the  following 
approach: 

(1)  For  monitoring  vital  signs  and  detecting  unsafeness,  use 

a  "hardware  monitor"  (one  per  patient). 

(2)  For  doctor  notification,  locate  a  "ward  station"  in  each 

ward.  The  nurse(s)  will  have  responsibility  for 
patients  In  his/her  ward  and  will  notify  the  doctor 
on  any  unsafe  patient  condition  as  reported  by  the 
ward  station. 


Figure  3.3. 1-5  shows  PMS  functionality  allocated  to  hardware 
monitors  and  ward  stations.  Figure  3.3. 1-6  gives  partial 
characterizations  of  e  hardware  monitor,  a  ward  station,  and  how 
they  constitute  a  PMS  solution.  The  notation  employed  Is  the  same 
as  in  figure  3.3. 1-2.  A  (probably  manual)  check  will  verify  that 
the  composition  of  their  activities  Is  a  correct  refinement  of  PMS 
activities  of  figure  3.3. 1-2. 


ACTIVITY  PRODUCES  VITAL  SIGNS 


DERIVES  VITAL  SIGNS  RECEIVED  El  PMS. 

ACTIVITY  GIVES  HEALTH  SIGNS 

USES  STATUS  CHECK  GENERATED  EX  NURSE. 
DERIVES  HEALTH  SIGNS  RECEIVED  EX  NURSE. 


VIEWPOINT  PMS 

Activity  Detect  unsafe  vital  signs 

USES  VITAL  SIGNS  GENERATED  EX  PATIENT. 

DERIVES  ENERGENCY  NOTIFICATION  RECEIVFP  EX  NURSE. 


Viewpoint  nurse 

Activity  Detects  unsafeness 

USES  HEALTH  SIGNS  GENERATED  EX  PATIENT 
DERIVES  DOCTOR  CALL  RECEIVED  BX  DOCTOR. 

Activity  Fires  doctor 

USES  ENERGENCY  NOTIFICATION  GENERATED  EX  PMS. 
DERIVES  DOCTOR  CALL  RECEIVED  BX  DOCTOR. 

Activity  check  patient 

DERIVES  STATUS  CHECK  RECEIVED  BX  PATIENT 

VlEWPOJM  Doctor 

Activity  Treat  Patient 

USES  DOCTOR  CALL  GENERATED  EX  NURSE. 


Figure  3.3. 1-2  Domain  Model:  Role  characterizations. 


System  Viewpoints 


Patient  I  PMS  Nurse!  Doctor 


HardwarF'T  Ward 

MONITOR  STATION 


Figure  3.3. 1-5  First  Solution  -  Refinement  of  PMS  Viewpoint. 


unt  Hardware  monitor 
Activity  Detect  unsafe  vital  signs 
uses  vital  signs  ... 

derives  unsafe  corciTiON  received  Ward  station 
ATTRIBUTE  ESEfl£SO  =  12  SECONDS 
DURATION  =  3  SECOrCS 
OPERATIONAL  COST  =  S500/PAT I  ENT/DAY 

unt  Ward  Station 
Activity  Set  alarm 

USES  UNSAFE  CONDITION  GENERATED  EX  HARDWARE  MONITOR 
DERIVES  ENERGENCY  NOTIFICATION  RECEIVED  £1  NURSE. 

Activity  Passes  patient  request. . . 

Figure  3.3. 1-6  First  Solution  -  Extenoed  Domain  Model 
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To  benefit  from  consistency  and  completeness  checking,  the  domain 
model  of  figures  3.3. 1-2  and  3.3. 1-3  (designated  here  as  DM  I)  can 
be  extended  to  Include  figure  3.3. 1-6  characterizations. 

Equivalent  PMS  activities  should  then  be  deleted.  We  designate 
the  result  DM  1 1 .  In  this  way.  Goals,  Requirements,  and  Solution 
architectures  can  be  made  to  share  (though  different  versions  of) 
the  same  domain  model.  DM  I  reflects  the  domain  model  referenced 
by  the  Goals.  DM  II  has  a  particular  solution  '’hard-coded1'  In, 
but  versioning  preserves  DM  I,  facilitating  tracking  across 
solution  revisions  and  refinements. 


Note  under  activity  "detect  unsafe  vital  signs"  (figure  3.3. 1-6), 
the  inability  to  easily  express  which  ward  station  should  receive 
the  "unsafe  condition"  data.  The  lack  of  better  formalisms  leaves 
the  solution's  specification  ambiguous  and  appeal  must  be  made  to 
use  of  free  text. 


Analysis  of  the  operational  cost  must  be  performed  manually.  The 
goal  of  5 1  CO  per  patient  per  day  is  stated  in  figure  3.3.1  — ^ .  The 
cost  implied  by  the  solution  is  over  $500  per  patient  per  day, 
because:  (1)  the  solution  implies  that  all  patients  will  be 
hocked  up  to  a  hardware  monitor  every  day  of  their  stay,  and  (2) 
this  cost  clone  is  $500  per  patient  per  day  (figure  3.3. 1-6). 

Thus  another  solution  is  desired,  one  In  which  not  every  patient 
is  connected  to  a  hardware  monitor  every  day  of  their  stay. 

Ferhaps  frequent  nurse  visits  could  replace  use  of  a  hardware 
ronitor  most  patient  days.  This  suggests  the  next  solution 
attempt. 


Esyjcj 


Still  attempting  to  find  a  feasible  solution,  the  requirements 
engineer  tries  this  approach: 

(1)  Patients  In  Intensive  care  will  be  hooked  to  hardware 
mon itors. 


(2)  Remaining  patients,  "Fair  or  better",  will  be  regularly 

visited  by  nurses  on  rounds. 

(3)  All  patients  will  still  have  access  to  a  nurse  alarm. 

(4)  °MS,  l.e.  the  Ward  station,  will  Inform  the  nurse  of: 

A  critical  situation  (1  above). 

When  it's  time  to  check  patients  (2  above). 

Or  of  a  nurse  alarm  occurrence  (3  above), 
providing  a  form  of  patient  tracking  (addressing  goal  3, 
f  igure  3.3. 1-4) . 


Thus  two  different  kinds  of  patients  must  be  recognized.  This  is 
reflected  In  figure  3.3. 1-7  (PMS  functional ity  will  still  be 
allocated  between  Hardware  monitors  and  Ward  stations).  Their 
different  needs  are  character Ized  In  the  revised  domain  model 
Illustrated  In  figures  3.3. 1-8  and  3.3. 1-9,  in  which  the  solution 
Is  again  Incorporated.  Figure  3.3. 1-9  replaces  the  original 
patient  transaction  (figure  3.3. 1-3)  with  two  transactions,  one 
for  each  type  of  patient. 

It  Is  assumed  that  as  a  result  of  discussions  (with  doctor 
representat I ves,  etc.)  on  the  cost  Implications  of  Immediate 
notification  and  the  strategy  behind  the  revised  solution.  It  is 
determined  that  the  doctor’s  goal  of  Immediate  notification  on 
unsafeness  (figure  3.3. 1-4)  can  be  relaxed.  The  result  is  the  two 
(preliminary)  requirements  Indicated  In  f Igure  3.3. 1-10,  which 
reference  the  transactions  shown  in  figure  3.3. 1-9. 


Various  analyses  can  be  applied  to  the  solution  to  gain  insight 
into  its  appropriateness  and  feasibility.  Seme  of  the  analyses 
will  be  supported  by  planned  1990  RET  tools,  others  must  be 
performed  manually. 

Cost  Mat  vs  Is 

Carried  out  manually.  Can  goal  2  of  f igure  3.3. 1-4  be  met?  Let: 
I  =  number  of  intensive  care  patients,  average  per  day. 

F  =  number  of  other  patients,  average  per  day. 

Assumptions: 

(1)  Frequency  of  "Make  rounds  on  patient"  =  2  hours  (for 

each  fair  or  better  patient). 

(2)  Time  to  "Makes  rounds  on  patient"  =  2  minutes  (per 

patient) . 

(3)  Time  to  "Gives  health  signs"  =  1  minute. 

(4)  Time  to  "Detects  unsefeness"  =  2  minutes. 

(5)  Wage  of  Nurse  =  $25  per  hour  (loaded  hourly  rate). 

These  assumptions  Imply  a  fair  or  better  patient  operational  cost 
of  $25  per  patient  per  day  (because  12  checks  per  day  times  5 
minutes  per  check  (assumptions  (2)  -  (4))  *  60  minutes  per  day, 
and  1  hour  per  day  times  $25  per  hour  *  $25  per  day).  Assuming 
the  relative  number  of  patients  of  each  type  satisfies: 

(6)  F  >  5.3  *  I, 

the  operational  cost  goal  can  be  met. 


Intensive  care  patient 


Activity  produces  vital  signs 

DERIVES  VITAL  SIGNS  RECEIVED  HARDWARE  MONITOR 
...  (OTHER  ACTIVITIES  AS  IN  PATIENT  VIEWPOINT  FlG.  3.3. 1-2.) 

Viewpoint  fair  or  better  pat i ent 
Activity  gives  health  signs 

LISES  STATUS  CHECK  GENERATED  B 1  NURSE 
DERIVES  hEALTH  SIGNS  RECEIVED  fil  NURSE 

...  (OTHER  ACTIVITIES  (EXCEPT  "PRODUCES  VITAL  SIGNS")  AS  IN  PAT  I  ENT  VIEWPOINT 

Fig.  3.3. 1-2.) 

Viewpoint  Hardware  monitor 

Activity  detect  unsafe  vital  signs 

JJSES  VITAL  SIGNS  GENERATED  fi 1  INTENSIVE  CARE  PATIENT... 


"Notify  doctor  on  unsafe  intensive  care  patient" 

RELIABILITY  =  99%  OF  NOTIFICATIONS  ARE  REPORTED 
DURATION  =  30  SEGOMOS 
source  =  Doctor 


Transaction  "Notify  Doctor  on  unsafe  Fair-or-better  Patient" 

reliability  =  99%  of  notifications  are  reported 
duration  =  2  HOURS 
source  =  Doctor 

Figure  3.3.1-10  ^icxjirenents  -  Excerpt  Showing  Doctor's  Goal  Has  Been  Revish). 


Carried  out  manually.  Can  the  notification  requirements  (figure 
3.3.1-10)  be  met?  The  requirements  engineer  estimates  and 
allocates  time  to  the  activates. 

Assumpt Ions: 

(1)  Time  to  "Find  doctor"  (e.g.  by  beeper)  =  10  seconds. 

(2)  for  "Makes  rounds  on  patient":  If  nurse  Is  given  15 

minutes  advance  warning,  the  nurse  has  enough  free 
time  to  check  on  a  patient  within  that  time. 

(Note  that  a  check  takes  5  minutes  according  to  the 
cost  analysis  assumptions.) 

Constraints: 

(3)  Time  to  "Set  alarm"  =  5  seconds. 

(4)  Frequency  of  "Prompt  for  rounds"  =  2  hours  (per  patient). 

Issues  "prompt"  15  minutes  before  patient  must  next 
be  checked  on  a  round. 

Above,  (1)  and  (3)  ensure  the  first  notification  requirement  can 
be  met  (sum:  maximum  time  it  takes  to  "Detect  unsafe  vital  signs" 
(figure  3.3. 1-6)  +  duration  of  "Set  alarm"  +  duration  of  "Find 
doctor"  =  (12  +  3)  +  5  +  10).  Above,  (2)  and  (4)  ensure  the 
second  notification  requirement  can  be  met  (when  function  "Prompt 
for  rounds"  gives  a  prompt,  (4)  says  that  the  nurse  has  15  minutes 
to  check  the  patient  on  a  round,  but  (2)  says  that  15  minutes  is 
sufficient  to  ensure  the  check  takes  place). 


A  Note  on 


In  the  above  analyses,  "assumptions"  are  constraints  on  the  PMS 
environment  (e.g.  external  activities).  They  could  be  recorded  as 
attributes  or  annotations  in  the  domain  model.  "Constraints" 
above  refers  to  solution  architecture  constraints;  they  thus  form 
part  of  the  solution  architecture.  It  Is  convenient  to  have  the 
Solution  architecture  share  the  same  domain  model  with  Goals  and 
Requirements,  and  thus  such  constraints  can  be  recorded  as 
attributes  or  annotations  In  the  PMS  part  of  the  domain  model. 


Several  tools  support  prototyping.  Prototyping  addresses  the 
question  of  whether  PMS  Interface,  functionality,  and  performance 
requirements  are  correct  (appropriately  reflect  expected  usage  and 
needs).  For  example,  the  requirements  engineer  would  probably 
want  Nurse  representatives  to  validate  PMS  patient-tracking 
functionality,  and  perhaps  the  ward  station  display  as  well.  Thus 
he  might: 

(1)  Specify  an  executable  prototype  of  the  ward  station  to  aid 

validation  of  the  requirements  and  solution  approach  to  PMS 
patient-track ing. 


? 
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(2)  Prototype  the  ward  station  display  to  aid  validation  of  the 

PMS  Interface  display. 

(3)  Calculate  whether  the  timing  constraints  on  PMS  activities  can 

be  realized;  especially  the  PMS  "constraints"  documented 
during  cost  and  performance  analyses. 

In  the  1990  RET,  these  activities  will  be  supported  by  special 
too  I s . 

For  activity  (1),  figure  3.3.1-11  ill ustrates  a  portion  of  a 
prototype  definition  the  requirements  engineer  might  construct 
using  the  VHLL  Prototyping  tools.  This  tool  would  also  assist 
activity  (3). 

The  definition  of  the  Ward  Station  in  figure  3.3.1-11  is  given  as 
a  dataflow:  arcs  represent  data,  and  bubbles  represent  functions. 
The  dataflow  is  organized  as  fellows: 

*  Each  Ward  Station  activity  (e.g.  "Set  Alarm")  is  represented  as 

an  independent  horizontal  dataflow. 

*  Each  Ward  Station  state  (e.g.  "Patient  Database")  Is  represented 

by  a  vertical  grouping  of  all  bubbles  that  operate  on  it. 

The  bubbles  in  such  a  grouping  cannot  execute  In  parallel. 

*  Labeled  boxes  represent  reference  data:  boxes  labeled  "A" 

represent  hardware  monitor  signals,  and  boxes  labeled  "B" 
represent  the  patient  database.  Reference  data  does  not 
cause  activation  of  bubbles. 

*  Each  bubble  has  associated  code.  An  annotation  beneath  the 

bubble  Indicates  a  bound  on  execution  time;  for  example 
"<  2"  means  execution  takes  less  than  2  seconds. 

The  result  is  a  prototype  fer  activity  (1).  Ignoring  resource 
contention,  we  see  that  the  5-second  constraint  on  "set  alarm", 
obtained  by  the  previous  performance  analysis,  can  be  met  If  timer 
frequency  F^  can  be  made  sufficiently  small  (e.g.  1  second 
frequency).  This  assists  activity  (3). 

For  activity  (2),  figure  3.3.1-12  Illustrates  the  screen  portion 
of  an  Interface  the  requirements  engineer  might  construct  using  an 
RPS  tool.  Asterisks  Indicate  comments  elicited  from  the  nurse 
when  exercising  the  Interface.  The  RPS  tools  also  assist  activity 
(3). 

Stress  scenario 


Another  RPS  tool  could  be  used  to  construct  a  canned  scenario  to 
exercise  the  proposed  solution  architecture.  Part  of  a  scenario 
definition  is  depicted  In  f igure  3.3. 1-13. 
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Figure  3.3.1-13 


PMS  Stress  Scenario  -  Excerpt. 
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In  figure  3.3.1-13,  all  times  are  In  seconds  after  the  scenario 
start.  In  the  scenario,  two  hardware  monitors  detect  unsafe  vital 
signs  at  12  seconds.  Both  take  3  seconds  to  signal  an  unsafe 
condition  to  the  Ward  Station.  Thus  at  15  seconds,  the  Ward 
Station  must  deal  simultaneously  with  two  emergency  notifications 
and  two  Falr-or-better  patients'  requests  for  nurse  assistance. 

Analyses  results 

The  primary  result  of  these  analyses  is  a  determination  of  the 
soundness  of  the  solution  approach. 

If  fundamentally  wrong,  a  new  solution  approach  must  be  attempted, 
or  the  requirements  revised  (which  might  require  extensive 
discussions  (both  consultation  and  negotiation)  with  mission 
users) . 

Otherwise,  the  additional  assumptions,  constraints,  and 
modifications  revealed  during  the  analyses  must  be  added  to  the 
Requirements  and  Solution  arch itecture.  Often  such 
additions/modifications  will  be  in  the  form  of  added/modified 
attributes  and  annotations  to  their  shared  domain  model  (i.e.  the 
version(s)  associated  with  the  solution  approach). 

In  our  scenario,  we  assume  ideal  results  of  our  tool-supported 
analyses  and  that  no  further  revisions  and  analyses  are  required. 
However,  it  is  appropriate  to  include  some  of  the  results  of  these 
analyses  into  the  Final  Requirements  and  Solution  Architecture 
documentation  ("final"  as  far  as  the  RET  exercise  is  concerned). 
For  example:  (1)  documentation  of  a  I  I  prototyping  exercises 
(objectives,  requirements  interpretation,  design,  solution 
analyses,  exercises  against  scenarios,  and  results),  (2)  screen 
mockups  (e.g.  figure  3.3.1-12),  and  (3)  stress  scenarios  for  later 
incorporation  into  documentation  on  acceptance  and  validation 
testing. 

Figure  3.3.1-14  gives  an  excerpt  of  the  final  Requirements  and 
Solution  architecture,  indicating  the  changes  that  need  to  be  made 
as  a  result  of  the  new  assumptions  and  constraints  revealed  during 
analyses. 

The  recujlremejvfs  engineer's  conclusions  on  tJi.e..RET.  exercise. 

What  might  the  requirements  engineer  conclude  from  the  RET 
exerc I se? 

*  Original  goals  conflicted,  namely  low  operational  cost 
vs.  rapid  doctor  notification. 

*  A  compromise,  achieved  thru  moderating  the  rapid  doctor 
notification  goal,  but  not  the  low  cost  goal,  led  to  an 
acceptable  set  of  requirements. 


Activity  Finds  doctor 


Attribute  Duration  =  10  sEcoros 

Activity  Makes  roods  on  patient 
Attribute  Frequency  =  2  hours 
Duration  =  2  minutes 

15  MINUTES  ADVANCE  WARNING  SUFFICIENT  TO  ENSURE  CHECK 

Activity  Detects  unsafeness 

Attribute  Duration  =  2  minutes 
Average  loaded  wage  =  $25/hdur. 

(2)  Viewpoint  Patient 

Activity  Gives  health  signs 

Attribute  Duration  =  1  minute 

LET  I  =  AVERAGE  #  OF  INTENSIVE  CARE  PATIENTS  PER  DAY, 

F  =  AVERAGE  #  OF  FAIR-OR-BETTER  PATIENTS  PER  DAY, 

Then  F  >  5.3  *  I 

EQUIRENENTS  ON  PMS  ACTIVITIFS 

(1)  Activity  Set  alarm 

Attribute  Duration  =  5  secohos 

(2)  Activity  Propt  for  roods 

Attribute  Frequency  =  2  hours 

ISSUES  THE  PROMPT  15  MINUTES  BEFORE  PATIENT  MUST  NEXT  BE  CHECKED  ON  A  ROOD 

Figure  3,3.1-14  Final  Requirements  &  Solution  Architecture  -  Om_y  ffcw  Changes  Shown. 


*  Pf-'S  functionality  and  performance  requirements  were 
defined  and  validated. 

*  A  feasible  solution  approach  was  defined. 


It  is  important  to  the  RET  that  these  conclusions  be  carefully 
documented  and  linked  to  the  Engineering  Context  Description. 

These  will  aid  determination  of  the  effectiveness  of  RET  tools  and 
techniques. 


3.2  19.95  Scenario 

The  1995  scenario  illustrates  these  technologies/capabilities: 

*  A  single  language  for  expressing  Goals,  Requirements  and 

Solution  architectures,  with  these  capabilities: 

Shared  Domain  Model  -  expressions  such  as  goals, 

requirements,  and  solutions  can  all  reference  the 
domain  model  In  order  to  f ac i I itate  thei r 
statement. 

Formal  Interpretation  of  Goals  and  Requirements  as 
predicates  against: 

Solutions  -  e.g.  cost  analysis 
Behavior  generated  by  executing  the  Solution 
against  scenario(s) ). 

*  Multiple  I  eve  I s  of  abstract  ion  -  there  must  be  a  capability 

of  tracking  refinements  across  levels. 

*  Incremental  evolution  of  requirements. 

*  Automated  generation  of  scenario  -  generated  from  a  mission 

user's  outl Ine  of  the  desired  behavior. 

To  summarize,  the  major  theme  of  the  1995  scenario  Is  that  the  use 
of  a  single  formal  language  permits  significant  automation  of  many 
activities  (e.g.  cost  analysis,  comparison  of  requirements  to 
generated  behaviors,  scenario  generation). 

Warning:  it  is  difficult  to  predict  what  notations  might  be 
employed  in  1995.  Thus  the  1995  scenario  freely  uses  a  formal 
Engl lsh-1 Ike  notation  to  convey  to  the  reader  what  is  being 
expressed  -  but  not  how  it  might  be  expressed.  It  Is  doubtful 
that  1995  technology  could  support  such  a  notation. 

Recalling  figure  3.3-1  (minus  the  third  wish),  we  begin  the  1995 
scenario.  As  In  the  1990  scenario,  the  hypothetical  RET  user  is 
called  a  "requirements  engineer". 
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From  user  Interviews  and  existing  documentation,  the  requirements 
engineer  creates  a  domain  model  of  the  operational  environment  of 
the  target  system  (PMS).  In  1995,  creation  of  high-fidelity 
models  will  be  possible.  Figure  3. 3. 2-1  gives  only  a  partial 
description.  Illustrating  what  such  a  domain  model  might  be  like. 
In  the  requirements  engineering  process  model,  such  a  domain  model 
is  part  of  (or  referenced  by)  the  goals  description. 

Figure  3. 3. 2-2  illustrates  the  goals.  The  objective  of  the 
requirements  engineering  exercise  is  to  analyze  the  trade-off 
between  performance  and  cost  wishes.  These  wishes  have  now  been 
formally  stated  as  goals.  The  goals  reference  Information 
contained  In  the  domain  model. 

The  first  goal  of  figure  3. 3. 2-2  is  really  two  goals:  a  logical 
goal  that  some  doctor  be  notified  and  a  performance  goal  that  such 
notification  be  Immediate  (the  meaning  given  to  "whenever").  To 
refine  the  logical  goal  In  order  to  say  which  doctor,  we  need 
terminology  (etc.)  that  would  come  from  a  better  domain  model  of 
PMS  context.  Figure  3. 3. 2-3  illustrates  the  resulting  refined 
model  and  refined  goal.  Figures  3. 3. 2-1  and  3. 3. 2-2  represent  a 
high  level  of  abstraction.  Figure  3. 3. 2-3  represents  a  deeper 
level  cf  abstraction  and  better  fidelity.  By  1995,  language 
mechanisms  will  exist  that  permit  stating  such  refinements 
explicitly  (incremental  modification)  and  there  will  be  automatic 
checks  for  consistency. 

To  determine  feasibility,  the  requirements  engineer  will  now  treat 
the  goals  as  preliminary  requirements  and  consider  what  kinds  of 
solutions  are  available  to  him. 


The  requirements  engineer  needs  to  consider  how  to  build  a  system 
that  will  satisfy  the  requirements. 

Suitable  resources  (hardware,  software,  or  people)  must  be 
Identified.  For  example,  consider  a  "Hardware  monitor",  figure 
3. 3. 2-4. 

The  domain  model  must  be  extended  to  include  such  a  resource  and 
constraints  on  how  it  might  be  used  so  that  formal  solutions  can 
be  expressed.  For  example,  see  figure  3.3. 2-5,  which  Illustrates 
how  the  Hardware  monitor  Is  brought  Into  the  domain  model  by 
extending  the  latter. 

Now  the  requirements  engineer  can  state  the  solution.  In  our 
example,  every  patient  is  hooked  to  a  hardware  monitor  which  Is 
then  hooked  to  his  doctor  (figure  3. 3. 2-6). 


$ 


Hardware  Monitor  is  a  device: 
Hospital  type  oe  Institution; 
Patient  type  oe  Person/ 

admitted-to  Hospital; 
Doctor  type  ce  Employee, 

Works-for  Hospital; 


Figure  3 .3. 2-1  Domain  Model 


*  Whenever  Unsafe  (some  Patient), 

Warn-Doctor  (some  Doctor,  that  Patient); 

*  Cost  <$ 100/Patient/Day 


Figure  3. 3. 2-2  Goals 


Hospital  has  a  set  of  Wards; 

Patients  are  assigned  iq  a  PARTICULAR  ward; 
Each  ward  always  has  a  doctor  on-duty; 


Eos  ANY  patient  that  needs  a  doctor, 

USE  THE  DOCTOR  THAT  IS  ON-DUTY  IN  IHE  PATIENT'S 
WARD . 


Figure  3. 3. 2-3  Refined  Model  and  Goals 


Hardware  Monitor  is  a  device; 

Every  j £  seconds,  u  CHECKS  ihe  vital  signs  qf  a  single 

PATIENT; 

H  ISSUES  AH  ALARM  IE  IH£  PATIENT  IS  UNSAFE; 

C&SJ  IS  S500/PAT  I  ENT/DAY; 

Figure  3. 3. 2-4  A  Resource 


Patient  has  vital  signs; 

Hardware-monitor  samples  vital  signs; 

Hardware-monitor  determines  whether  patient  is  unsafe; 
Connect  hardware-monitor  alarm  iq  doctor,  use  IQ  notify 

HIM; 

Figure  3. 3.2-3  Extending  the  Domain  model 
to  Include  the  Resource. 


Eqk  Each  Patient: 

Hook  patient  iq  hardware  monitor 
Hook  hardware-monitor  alarm  iq  doctor 

Figure  3. 3 .2-6  A  Candidate  Solution 


Cost  =  $500/pati ent/day 
Cost  exceeds  Goal 


Figure  3. 3. 2-7  Automatic  calculation  of 

Solution  Cost  and  Evaluation 
against  Goals  and 
Requirements. 


Extending  the  domain  model  to  formally  Include  the  resource  and 
constraints  (as  Indicated  above)  facilitates  automatic  formal 
analyses  of  solutions.  There  are  at  least  two  types  of  analyses 
that  can  be  done:  (1)  detection  of  Illegal  or  Incomplete 
solutions,  etc.,  and  (2)  determination  of  operational  parameters 
such  as  cost,  downtime,  etc.  Formal  Interpretation  of 
Goals/Requirements  as  predicates  against  the  results  of  type  2 
analyses  lead  to  an  automatic  determination  of  whether  the 
solution  Is  satisfactory  In  certain  ways. 

For  our  PMS  example,  consider  a  tool  or  analysis  that 
automatically  calculates  operational  cost  and  formally  Interprets 
Goa  I s/Requ I rements  as  predicates  against  that  cost.  Figure  3.3.2- 
7  illustrates  what  happens.  The  resulting  operational  cost  does 
not  satisfy  the  cost  goal  (figure  3. 3. 2-2)  and  the  requirements 
engineer  is  so  informed. 
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sn.erat.icn  .and  Analysis 


The  f c I  lowing  will  be  a  1995  strategy  to  test  a  solution  for 
satisfactory  performance,  it  will  be  largely  automated  (relative 
to  what  can  be  done  even  in  1990):  (1)  create  Scenarios,  (2) 

formally  interpret  Goals/Requirements  as  predicates,  and  apply  the 
predicates  against  the  behavior  elicited  when  executing  the 
sclution  on  the  scenarios,  and  (3)  note  which  Goals/Requirements 
are  not  satisfied.  In  the  1995  RET,  (2)  and  (3)  will  be 
ccmpletely  automatic;  (1)  will  be  significantly  automatic. 

Simple  scenarios  can  be  automatically  generated.  In  our  PMS 
example,  if  the  requirements  engineer  requests  "Build  scenario  for 
a  patient",  two  scenarios  would  be  generated,  one  for  a  patient 
who  at  some  point  becomes  "unsafe"  (see  figure  3. 3. 2-8)  and  one 
for  an  always  safe  patient.  In  the  general  case  of  scenario 
generation,  the  requirements  engineer,  by  indicating  key  events, 
will  control  the  combinatorial  explosion  of  scenarios  that  might 
be  generated  from  considering  all  cases.  For  example,  "Build  a 
patient  scenario  which  Includes  'patient  becomes  unsafe'". 

In  our  PMS  example,  the  running  of  the  solution  (figure  3. 3. 2-6) 
against  the  scenario  (figure  3. 3. 2-8)  produces  behavior  that  can 
be  tested  against  the  predicates  resulting  from  a  formal 
Interpretation  of  the  goals  (figures  3. 3. 2-2,  3),  yielding  the 
conclusions  Indicated  In  figure  3. 3. 2-9.  Recall  that  the  first 
goal  In  figure  3. 3. 2-2  Is  Interpreted  as  both  a  logical  goal  (that 
notification  takes  place)  and  a  performance  goal  (notification 
must  be  immediate).  Figure  3. 3. 2-9  says  that  the  notification 
goal  was  satisfied,  but  not  the  performance  goal,  because  the 
hardware  monitor  is  a  sampling  device  (figure  3. 3. 2-4). 


* 


Patient  admitted  to  hospital; 
Patient  assigned  to  ward; 


*  Patient  hooked  to  hardware  monitor; 

Doctor  hooked  to  alarm; 

*  Hardware  monitor  samples  patient  vital  signs; 

Patient  Unsafe: 

Alarm  issued; 

On-duty  DOCTOR  WARNED; 

Figure  3. 3,2-8  Generation  of  a  Scenario 

Depicting  an  Unsafe  Patient. 


Goal  of  notification  when  patient  unsafe 

==>  SATISFIED 

Goal  of  immediate  notification  when  patient  unsafe 

==>  MOT  SATISFIED 


Figure  3. 3. 2-9  Automatic  analysis  of  the 
result  of  Executing  the 
Solution  against  the 


The  conclusion  of  this  formal  analysis  Is  that  the  solution 
satisfies  the  notification  goal  but  not  the  performance  or  cost 
goals.  Thus  another  solution  approach  must  be  found  that  makes 
these  goals  realizable,  or  the  goals  must  be  revised. 

Deriving  Requirements 

We  assume  that  further  Interviews  etc.  reveal  that  significant 
delays  In  notifying  the  doctor  are  acceptable  If  the  patient  Is 
noncritlcal,  so  the  goals  can  be  revised.  Such  patients’  safeness 
can  be  monitored  by  a  nurse  on  rounds.  The  requirements  engineer 
thus  extends  the  domain  model  by  Introducing  the  concepts  of 
intensive  care  and  nurse,  and  derives  two  performance  requirements 
as  Indicated  In  figure  3.3.2-10.  Note  that  the  performance 
requirement  for  intensive  care  patients  has  been  relaxed  to  equal 
the  sampling  rate  of  the  hardware  monitor. 

F.ev.is.ing  SoJu.t.ioA  ..arch.itect.uiie 

Next,  the  requirements  engineer  attempts  to  use  the  nurses  to  find 
a  solution  that  makes  the  requirements  feasible,  including  cost. 
Parts  of  such  an  attempt,  solution  and  extended  dona i n  model,  are 
indicated  in  figure  3.3.2-11.  The  nurse  not  only  dees  rounds,  but 
also  monitors  the  hardware  alarm. 

As  with  the  previous  solution  ( f igure  3. 3. 2-6) ,  the  following 
analysis  is  done  on  the  solution:  cost  analysis  and  a  check  for 
satisfiability  with  the  requirements  through  execution  against 
scenarios.  We  assume  the  cost  analysis  reveals  that  the  current 
solution  satisfies  the  <  $100/patient  goal.  We  also  a:  ume  that 
the  satisfiability  check  reveals  that  Introduction  of  a  nurse 
creates  a  delay  for  the  Intensive  care  patient  case  (because  there 
Is  nc  longer  a  direct  connection  from  alarm  to  doctor).  This 
means  we  need  to  find  another  solution  that  will  make  the 
requirements  feasible,  or  revise  the  requirements. 

BfiylSfid.  Requirements.  (Demonstration  of  Incremental  Modification) 


We  assume  we  revise  the  requirements.  Figure  3.3.2-12  shows  the 
explanation  that  might  be  given  for  the  modification  to  the 
Intensive  care  15-second  notification  requirement. 


All  patients  in  intensive  care  have  hardware  monitors 
Notification  occurs  within  15  seconds; 

All  other  patients  are  monitored  by  a  nurse  on  rounds 
Notification  occurs  within  2  hours; 


Figure  3.3.2-10  Requirements 


Nurse  type  qe  Employee, 

Works-for  Hospital; 

Nurse  monitors  hardware  alarm; 

Nurse  phones  on-duty  doctor  ie  alarm  is  act.i 


Figure  3.3.2-11  Candidate  Solution. 


PANEL  RESEARCH  AND  DEVELOPMENT  RECOMMENDATIONS 


Below,  we  summarize,  by  source,  the  RET  R&D  program  objectives: 

*  1990  and  1995  Scenarios  -  the  R&D  program  should  provide 

tools  and  methods  that  work  together  In  the  ways  Illustrated 
by  the  scenarios  (section  3.3).  The  tools  and  methods 
should  have  the  capabilities  Illustrated  by  the  scenarios. 

*  Requirements  Engineering  Process  Model  -  The  R&D  program 

should  provide  tools  and  methods  to  support  process  model 
activities  (sections  3.1  and  3.2;  detail  Is  found  In 
appendix  C) . 

*  RADC's  plans  and  goals  -  The  R&D  program,  shou  Id  help  fulfill 

these  goals  for  the  RET  (section  2):  (1)  The  RET  should 

support  evaluation  of  the  effectiveness  of  tools  and 
methods.  (2)  The  RET  should  make  a  full  range  of 
requirements  engineering  capabilities  accessible  to  Air 
Force  mission  users  and  acquisition  engineers.  (3)  The  RET 
should  host  the  currently-contracted  tools,  and  by  1990, 
they  should  be  integrated. 

*  Long-range  architecture  for  the  RET  -  The  R&D  program  should 

realize  the  panel's  vision  of  an  RET  architecture  featuring 
(appendix  D):  (1)  a  direct  manipulation-style  user 

interface  to  all  objects,  (2)  a  database  serving  as  the 
common  repository  for  all  requ I rements-rel ated  information, 
and  (3)  a  formal  language  for  expression  of  goals, 
requirements,  and  solution  arch itectures.  ^ 

In  the  long  term,  all  RET  tools  and  methods  should  be 
structured  to  fit  this  architecture. 


References  are  made  to  these  objectives  In  the  sections  that 
follow.  Section  4.1  summarizes  the  panel's  strategy  for  obtaining 
an  Integrated  RET.  Section  4.2  discusses  an  R&D  program 
consisting  of  two  tracks:  (1)  an  Evolutionary  Track  for 
developing  tools  and  methods  that  in  the  near  term  provide  the 
best  payoff  In  better  requirements;  and  (2)  a  Formal  Language 
Track  for  exploring  the  higher  risk/payoff  implications  of  a 
formal  requirements  language.  Section  4.3  discusses  the  panel's 
recommendation  on  the  relative  allocation  of  R&D  resources  between 
these  two  R&D  tracks,  and  the  relative  prioritization  of  issues 
within  each  track. 


To  address  RADC's  objective  of  Integrating  the  currently- 
contracted  tools,  the  panel  recommends  that  integration  be 
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achieved  by  having  the  tools  work  off  a  common  database  and  be 
accessed  through  a  common  user  interface.  This  level  of 
Integration  means  that:  (1)  tools  can  share  date,  and  (2)  the  RET 
user  is  given  uniform  access  to  tools  and  their  data  and  is  free 
to  invoke  tool  functions  in  an  order  natural  to  his/her 
app I icat ion . 

This  approach  will  produce  an  early  version  of  the  long-range  RET 
arch Itecture. 

An  Integrated  RET  will  also  help  in  the  evaluation  of  tools;  for 
example,  by  providing  the  basis  for  a  broader  range  of  control 
exper  tments. 

To  significantly  reduce  the  amount  of  effort  required  to  achieve 
Integration,  the  panel  recommends  a  near-term  strategy  of 
standards  and  cooperation  between  the  RADC  tool  contractors.  The 
integration  strategy  is  further  discussed  in  appendix  D. 

To  provide  RET  users  some  of  the  benefits  of  integration  in  the 
very  near  term,  the  panel  recommends  a  loose  coupling  of  the 
current  I y-contracted  tools.  The  loose  coupling  plan  is  also 
discussed  in  appendix  D. 


4.2.1 


To  meet  the  objectives  stated  at  the  beginning  of  section  4,  the 
panel  identified  two  themes  on  which  RET  R&D  program  efforts 
should  focus:  (1)  providing  near-term  support  for  these 
activities:  prototyping,  requirements  analysis,  and  evaluation  of 
tools;  and  (2)  a  formal  treatment  of  requirements.  The  R&D 
program  consists  of  two  tracks  to  deal  with  these  two  themes. 
Figure  4.2. 1-1  depicts  the  two  themes  and  their  relationships. 
Below,  we  expand  on  these  themes  and  their  associated  activities 
and  then  discuss  the  figure. 


Both  the  1990  scenario  and  the  process  model  characterization 
demonstrate  the  Importance  of  prototyping  and  the  role  of 
scenarios  in  driving  prototypes.  (For  examples,  refer  to  figures 
3.3.1-11,  3.3.1-12,  and  3.3.1-13.)  Prototyping  gives  a  mission 
user  "vtsib i I Ity"  into  specifications  of  system  and  software 
requirements  by  helping  the  mission  user  determine  whether  his/her 
needs  are  being  addressed.  Thus  the  panel  recommends  that  the 
"creation  of  prototypes  and  scenarios,  and  analysis  of  results" 
activity  be  an  early  focus  of  the  RET  R&D  program. 
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Some  of  the  objectives  stated  in  section  4  imply  the  need  fcr 
tools  and  methods  that  address  the  non-solution-architecture 
phases  of  requirements  engineering;  specifically,  goals  and 
requirements  synthesis  and  analysis.  Such  tools  and  methods  would 
help  mission  users  state  their  needs  and  decisions  at  the  mission 
level.  Both  scenarios  demonstrate  the  need  for  such  capabilities. 
Such  a  need  must  be  satisfied  in  the  near  term.  Thus,  the  panel 
recommends  that  the  "analysis  on  requirements"  activity  be  another 
early  focus  of  the  RET  R&D  program. 

Evaluating  the  effectiveness  of  RET  tools  and  methods  requires 
the  capability  to  track  their  use  and  collect  results.  An 
integrated  RET  would  help  control  Independent  parameters  (e.g. 
style  of  presentation,  format  of  input  data),  a  prerequisite  for 
parallel  experiments.  Thus  the  panel  recommends  that  the 
"measurement  of  fools  in  an  integrated  RET"  activity  also  be  an 
early  focus  of  the  RET  R&D  program. 


To  successfully  provide  near-term  support  for  the  three  activities 
above  requires  a  lew  risk  and  early  payoff  strategy.  In  the  long 
term,  the  resulting  RET  capabilities  would  be  enhanced/refined. 

The  panel  organized  an  "Evolutionary  Track"  of  R&D  efforts  to  do 
this.  The  Evolutionary  track  would  provide  the  capab i I  it i es 
illustrated  In  the  1990  scenario  and  support  most  process  model 
activities.  The  Evolutionary  Track  is  described  In  section  4.2.2. 


The  panel  recognized  early  on  that  representing  requirements  in  a 
formal  language  was  a  high, -payoff  approach,  but  such  an  approach 
wculd  fail  to  provide  near-term  solutions  to  the  R&D  program, 
objectives.  Nevertheless,  such  an  approach  would  address  these 
objectives  not  being  addressed  in  the  other  track:  (1)  help 
automate  requirements  traceability  and  assessment  of  requirements 
coverage,  and  (2)  provide  a  language  for  representing 
requirements,  solution  architectures,  and  goals;  a  major  element 
of  the  long-range  RET  architecture.  The  panel  thus  defined  a 
"Formal  Language  Track"  whose  focus  would  be  to  provide  such  a 
formal  language. 

The  Formal  Language  Track  would  also  provide  the  capabilities 
Illustrated  In  the  1995  scenario  and  support  most  process  model 
act  I  v It ies. 


Figure  4.2. 1-1  depicts  1990  and  1995  goals  of  the  two  themes: 
creation  of  prototypes  and  scenarios  and  analysis  of  results 
( "Prototyp ing") ,  analysis  on  requirements  ("Analysis"), 
measurement  of  tools  in  an  Integrated  RET  ("Tool  Evaluation"),  and 
formal  language.  Their  dependencies  with  each  other  and  with  the 
current  I y-contracted  tools  are  indicated  by  the  arcs. 
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Analyst  tool  capabilities  are  the  basis  for  the  1990  Analysis 
capabilities  for  structuring  requirements  and  the  domain  model. 
Domain  models  provide  necessary  Information  for  building  scenarios 
and  simulations,  thus  the  vertical  dependency  with  1990 
Prototyping  goals.  1990  Prototyping  goals  are  also  dependent  on 
the  prototyping  and  scenario  generation  capabilities  provided 
respectively  by  the  VHLL  and  RPS  tools. 

1995  Prototyping  goals  extend  1990  Prototyping  capabilities  by 
providing  capabilities  for  sensitivity  analysis  on  requirements. 
1995  Analysis  goals  extend  1990  Analysis  capabilities  by  providing 
capabilities  for  dynamic  analysis  and  quality  critiquing.  Various 
Interactions  are  possible  between  1995  analysis  tools  for  these 
two  activities,  hence  the  double-headed  vertical  arrow. 

The  1990  Tool  Evaluation  standards  will  be  influenced  by  the 
standards  adopted  by  developers  of  the  currently-contracted  tools. 
These  1990  standards  will  in  turn  guide  all  subsequent  RET  tool 
development.  The  1995  Tool  Evaluation  goal  is  to  integrate  these 
new  Analysis,  Prototyping,  and  Formal  Language  capabilities  Into 
the  instrumented  RET  to  facilitate  their  evaluation. 

The  1990  Formal  Language  goals  are  independent  of  1990  Analysis 
and  Prctotyping  efforts  and  the  current  I y-contracted  tools.  The 
1995  Formal  Language  goals  Include  Incorporating  abstraction 
mechanisms  into  the  language  and  interpreting  classes  of 
scenarios.  These  capabilities  must  also  be  integrated  Into  the 
RET. 

From  the  perspective  of  milestones,  the  1995  RET  wil I  be  a  mature 
experimental  facility,  hosting  matured  analysis,  prototyping, 
formal  language,  end  evaluation  capabilities.  The  1990  RET  will 
be  a  prototype  of  the  1995  RET,  still  integrated  and  instrumented 
for  evaluation,  but  featuring  only  a  few  mature  tools,  in 
particular,  the  currentl y-contracted  tools. 


As  explained  above,  the  theme  of  the  Evolutionary  Track  focuses  on 
three  activities.  Below,  In  section  4.2.2. 1,  we  consider  these 
activities  In  turn,  indicating,  by  R&D  roadmap,  the  panel’s 
strategy  of  achieving  the  theme’s  1990  and  1995  goals.  In  section 
4. 2. 2.2,  we  consider  the  R&D  issues  that  constitute  the 
Evolutionary  Track. 

The  linkage  between  section  4. 2. 2. 2  R&D  issues  and  the  R&D  effort 
boxes  of  the  roadmaps  In  section  4.2.2. 1  is  Indicated  by  a  code(s) 
in  the  lower  right-hand  corner  of  most  boxes  in  the  roadmaps. 

This  code(s)  indicates  the  R&D  Issues  of  which  the  R&D  effort  Is  a 
part.  Below,  we  Indicate  the  correspondence  between  R&D  issue 
names  and  codes: 


CM :  Domain  Models  and  Information 

RSA:  Requirements  -  Static  Analysis 
RA;  Requirements  Analysts  Methodology 
DA:  Dynamic  Analysis 

SG:  Scenario  Generation  Support  4  Scenario  Coverage 

Analysis 

AR:  Scenario  Execution  and  Analysis  of  Results 

EC:  Estimation  of  Cost,  Risk,  Time  in  System  Development; 

Performance  4  Execution  Costs  Analysis 
RE:  Requirements  Evaluation 

TE:  Testbed  Effectiveness 

Ul :  User  I nterf ace 

DB:  Database 

ETI:  Evolutionary  Testbed  Integration 


4.2.2. 1  Objectives  and.  Roadmaps 
Requ i rements. Analysts 

The  goal  of  Requirements  Analysis  is  to  extend  and  automate 
capabilities  in  the  analysis  of  requirements.  Analyses  include 
analysis  for  consistency  and  completeness,  analysis  of 
interrelationships  among  the  requirements  and  with  domain  models 
and  scenarios,  and  quality  assessment:  detection  of  redundancy, 
determination  of  understandab i I  i ty  and  modifiability. 

The  parts  of  the  1990  scenario  addressed  include: 

Capabilities  for  partially  formal  characterizations  of:  the 
problem  domain  (including  resource  models),  functional 
requirements,  scenarios,  and  some  nonfunctional 
requirements.  Also,  static  analysis  capabilities  on  these. 

Capabilities  for  expressing  goals. 

Capabilities  logically  equivalent  to  those  indicated  In 
figures  3.3. 1-1  through  3.3.1-10,  the  Scenario  Event  Table 
in  f igure  3.3. 1-13,  and  figure  3.3.1-14. 

Figure  4.2.2. 1-1  indicates  the  panel's  strategy  of  meeting  the 
goal  of  this  activity.  The  roadmap  Indicates:  (1)  the 
formalisms,  tools,  methods,  etc.  that  support  the  above-mentioned 
(and  longer  range)  capabilities,  (2)  the  corresponding  R4D  efforts 
that  will  produce  these  formalisms,  tools,  methods,  etc.,  and  (3) 
the  dependencies  between  these. 


The  goal  of  Prototyping  is  to  extend  and  automate  capabilities  in 
the  creation  of  prototypes  and  scenarios  for  experiments,  and 
analysis  of  experimental  results.  Related  issues  include: 
creation  of  support  simulations,  coverage  analysis,  and  the 
presentation  of  experiment-generated  data. 

The  parts  of  the  1990  scenario  addressed  Include: 

Capabilities  for  partially  formal  characterizations  of: 
scenarios  and  their  Interfaces  (e.g.  Inputs)  to  prototypes. 

Capabilities  for  building  executable  models,  performance 
models,  and  scenarios.  Capabilities  for  exercising 
prototypes  against  scenarios.  These  capabilities  provide  an 
essential  basis  for  evaluating  feasibility  and  trade-offs. 

Capabilities  logically  equivalent  to  those  indicated  in 
figures  3.3. 1-3,  3.3. 1-9,  and  3.3.1-11  through  3.3.1-13. 

Figure  4.2.2. 1-2  indicates  the  panel's  strategy  of  meeting  this 
activity's  goal.  (In  the  figure,  two  boxes  have  wavy  left  edges. 
Such  a  left  edge  is  used  to  indicate  that  the  true  position  of  the 
edge  lies  further  to  the  left  but  can't  be  shewn  without  creating 
an  overlap  or  impacting  the  figure's  compactness.  The  true 
position  can  be  found  by  referencing  the  corresponding  R&D  issues 
i n  Appendix  E. ) 
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The  goal  of  Tool  Evaluation  is  to  provide  capabilities  for 
measuring  the  effectiveness  of  RET  tools  in  terms  of  improvements 
to  processes  (e.g.  productivity)  and  products  (e.g.  quality).  A 
related  goal  is  to  provide  an  integrated  RET  in  which  the  best 
features  of  each  tool  can  be  appl ied  to  the  same  set  of 
requ i rements. 

The  parts  of  the  1990  scenario  relevant  to  tool  evaluation 
Include: 

Establishing  RET  exercise  objectives,  documenting  results  of 
the  RET  exercise. 

The  tool  evaluation  context  might  appear  in  the  Engineering 
Context  Description,  establishing  objectives  for  and 
constraining  the  RET  exercise  for  the  RET  purpose  of  tool 
eval uat ion. 

Figure  4.2.2. 1-3  indicates  the  panel's  strategy  of  meeting  this 
act ivity's  goal . 


The  Evolutionary  Track  consists  of  both  research  and  development 
issues.  Appendix  E  gives  detailed  character izations  of  all 
Evolutionary  Track  issues.  Below,  we  provide  a  summary  of  the 
research  Issues  fol lowed  by  a  summary  of  the  development  issues. 

Characterization  of  Evolutionary  Track  Research  Issues 

The  research  Issues  generally  correspond  to  activities  In  the 
requirements  engineering  process  model.  This  Is  because  the  panel 
used  the  process  model  to  help  Identify  the  R&D  Issues  that  would 
help  realize  desired  RET  capabilities. 

The  Evolutionary  Track  R&D  Issues  are  considered  here  In  an  order 
roughly  corresponding  to  a  top-down  walk-through  of  the  process 
model . 

Each  research  Issue  is  characterized  in  terms  of:  where  It  fits 
in  the  process  model,  what  are  the  research  objectives,  what  is 
the  recommended  solution  approach,  and  recommendations  regarding 
funding  priority  based  on  assessed  risk  and  payoff.  The  funding 
priority  scheme  assigns  each  issue  a  priority  of  High,  Medium- 
high,  Medium,  Medium-low,  or  Low.  A  summary  of  the  prioritization 
is  given  in  section  4. 2. 4.1. 

GOALS  AND  REQUIREMENTS  SYNTHESIS 

Where  issue  fits  in  process  model:  In  transforming  wish  lists 
into  goals  and  transforming  goals  Into  requirements. 

Fesearch  Objectives:  Provide  capabilities  for:  (1)  expressing 
and  viewing  goals  and  requirements  and  (2)  expressing  goals  and 
requirements  in  one  or  more  domain-specific  languages  and 
Integrating  the  different  expressions. 

Solution  Approach:  Develop  mechanisms  for  syntax-directed 
editing,  view  management,  and  for  supporting  reuse.  Enhance  the 
Analyst  to  Interface  and  utilize  these  mechanisms. 

Recommendation:  Funding  priority:  Medium-low.  To  some  extent 
solution  mechanisms  can  be  provided  by  database  technology.  These 
should  be  brought  Into  the  RET;  In  which  case  the  probability  of 
success  Is  high,  and  no  specific  research  funding  Is  required. 


DOMAIN  MODELS  AND  INFORMATION 

Where  It  fits  In  process  model;  Domain  Information  is  essential 
to  development  of  goals,  requirements,  and  solution  architectures. 


Research  Objectives:  Provide  capabilities  to  col lect  and  organize 
domain  information  and  make  It  accessible  to  tools  and 
requirements  engineers. 

Solution  Approach:  Select  and  develop  mechanisms  for  capturing 
and  structuring  domain  information,  e.g.  knowledge  acquisition  and 
domain-specific  I anguages/ interfaces.  Build  interfaces  for  tools 
to  exploit  domain  Information.  Investigate  role  for  expert 
systems  that  use  domain  Information  to  help  elicit  and  validate 
requ I rements. 

Recommendation:  Funding  priority:  Medium-high.  Risk:  low  to 
moderate.  This  research  offers  much  promise,  but  many  of  the 
capabilities  it  attempts  to  provide  will  remain  manual. 

REQUIREMENTS  -  STATIC  ANALYSIS 


checking  capabilities. 


tss  mode  I :  Static  analysis  on  requ I rements. 
Provide  consistency  and  completeness 


So  I ut ion  Approach :  Define  a  controlled  (restricted  vocabulary) 
natural  language  for  the  expression  of  requirements.  Develop 
checkers  that  take  such  requirement  expressions  as  Input  and 
determine  their  consistency,  completeness,  etc.  with  respect  to 
meta-knowledge  bases. 


Recormendct icn:  Funding  priority:  Medium.  Risk:  I 
research  has  some  promise,  but  the  approach  is  fairl 
of  other  R&D  efforts. 


ow.  This 
y  independent 


REQUIREMENTS  ANALYSIS  METHOOOLOGY 


Where  it  fits  in  process  model:  Guides  the  order  in  which  process 
model  activities  are  carried  out. 

Research  Objective:  Develop  a  methodology  spanning  all 
requirements  engineering  activities  that  will  provide  guidance  in 
the  use  of  new  requirements  analysis  capabilities. 

Solution  approach;  Near  term:  Extend  the  CORE  method  Into 
solution  architecture  synthesis  and  analysis  and  provide  guidance 
In  use  of  RPS  and  VHLL  Prototyping  tools.  Long  term:  Select  a 
then-existing  methodology  that  better  meets  the  research  objective 
based  on  effectiveness  evaluation  of  existing  tools. 

Recommendation:  Funding  priority:  Medium-high.  Risk:  Moderate. 
Tools  must  be  evaluated  to  determine  their  range  of  effectiveness. 
This  research  has  much  promise. 
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DYNAMIC  ANALYSIS 

Where  It  fits  In  process  model:  In  dynamic  analysis  and  rapid 
prototyp I ng. 

Research  Objectives:  Provide  the  user  with  an  animation-based 
capability  (i.e.  graphically  depicting  flow  of  Information  and 
control)  to  investigate  the  consistency,  completeness,  and 
validity  of  a  set  of  requirements. 

Solution  Approach:  Enhance  prototyping  capabilities  with:  (1)  the 
ability  to  run  animated  exercises  of  the  proposed  system,  (2)  the 
ability  to  graphically  browse  through  requirements 
Interrelationships,  and  (3)  a  knowledge-based  enhancement: 
ability  to  represent  and  analyze  requirements  against  a  known 
domain  model . 

Recommendation:  Funding  priority:  Medium-high.  This  research 
offers  much  promise  at  generally  low  risk.  Payoff  and  risk  of 
knowledge-based  enhancements  Is  not  yet  clear. 


SCLlfT  I  CN  ARCHITECTURE  SYNTHESIS 


Where  it  fits  in  process  model:  Design  candidate  architecture. 


arch  objectives;  Develop  a  design  ass Istant  that  w 1  I  I  help 
construct  and/or  critique  a  solution  architecture. 


Soluticn  approach:  Develop  mete-models  of  design  goodness  end 
enhance  prototyping  tools  with  -knowledge  base  support  for  creation 
of  good  designs. 


basic  results 


. ;  Funding 
from  other 


priority:  Low. 
investigations 


Risk:  high.  Wait  for 
Into  the  "design  problem". 


SCENARIO  GENERATION  SUPPORT  &  SCENARIO  COVERAGE  ANALYSIS 


Where  It  fits  in  process  model:  Rapid  Prototyping  (building 
scenarios  that  drive  prototypes  and  determine  coverage). 


Research  objectives:  Provide  the  capability  to  build  adaptive 
scenarios  and  support-simulations  that  drive  a  prototype.  Provide 
the  capability  to  determine  which  parts  of  the  prototype  (and 
associated  requirements)  were  brought  into  play  during  execution. 

Solution  Approach:  Provide  a  knowledge-based  simulation  system 
that  can  interface  and  run  executable  and  performance  models  of 
the  target  system,  simulating  the  Interactions  between  the 
scenario  and  the  models.  Develop  dynamic  probes  Into  a  prototype. 
Investigate  coverage  by  proving  the  scenario  from  the  prototype 
description. 


Recommendation:  Funding  priority:  High.  Risk:  low  for  scenario 
generation,  higher  for  coverage.  Resulting  scenarios  should  be 
able  to  work  with  all  prototyping  tools. 


VALIDATION  OF  PROTOTYPE  AND  SCENARIOS 

Where  It  fits  In  process  model:  Rapid  prototyping. 

Research  Objectives:  Provide  the  capabl I Ity  to  val [date  the 
prototype  and  scenario  for  consistency,  completeness,  and  logical 
correctness.  Provide  the  capability  to  validate  the  results  of 
executing  the  prototype  against  the  scenario. 

So  I ut I on  approac  h :  Build  knowledge-based  syntax  and  semantics 
checker  for  prototypes  and  scenarios.  Create  a  library  of  metered 
and  validated  prototypes  that  would  be  used  to  gauge  the  accuracy 
of  the  results  of  a  particular  prototyping  exercise. 

Recommendation:  Funding  priority:  Low.  Much  of  the  necessary 
groundwork  (l.e.  reusability,  formal  expression  of  prototype  and 
scenarios)  for  validation  Is  being  addressed  by  other  research 
components  of  the  track.  Manual  validation  methods  will  continue 
to  be  necessary. 


SCENARIO  EX  EC  UT 1  ON  AND  ANALYSIS  OF  RESULTS 

Where  it  fits  In  process  model:  Rapid  Prototyping. 

Research  objectives:  Provide  tools  to  collect,  analyze,  and 
present  the  data  generated  during  prototype  experiments. 

Solution  Approach:  In  the  near-term:  a  database  to  hold  results 
and  data  management  facilities  to  aid  human  analysis;  each 
prototyping  tool  will  provide  a  specialized  analysis  capability. 

In  the  long-term;  knowledge-based  aids  for  evaluation  of 
prototype  sensitivity. 

Recommendation:  Funding  priority:  High.  Risk:  near-term:  low; 
long-term:  moderate  to  high.  Long-term  effort  assumes  an 
Integrated  RET  In  which  one  scenario  can  drive  multiple  models  and 
results  can  be  correlated. 


ESTIMATION  OF  COST,  RISK,  TIME  IN  SYSTEM  DEVELOPMENT;  PERFORMANCE 
&  EXECUTION  COSTS  ANALYSIS 

Where  It  fits  In  process  model:  Analysis  of  performance  and 
reliability,  and  development  cost,  risk. 

Research,  objectives:  Provide  metrics-based  capabilities  for  the 
estimation  of  cost,  time  and  performance  as  a  basis  for  making 
trade-offs  and  doing  impact  analysis. 
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Solution  approach:  Provide  metrics  for  cost,  time,  risk,  and 
project  size,  and  provide  related  tools  that  do  the  measurement 
and  analysis.  Add  metrics  for  d Istr Ibuted/know ledge-based 
systems.  Develop  tools  that  do  fault-handling  analysis  and 
reliability  estimation. 

Recommendation:  Funding  priority:  High.  Risk:  Moderate. 
Metrics  are  often  subjective,  thus  special  attention  Is  needed. 
Research  has  great  promise  supporting  critical  trade-off  and 
Impact  analysis  activities. 


REQUIREMENTS  EVALUATION 

Where  It  fits  in  p.-ocess  models:  Primarily  requirements 
evaluation  and  reformulation,  but  also  static  analysis,  solution 
architecture  synthesis  and  prototyping. 

Research  objectives:  Provide  capabilities  to  determine 
requirements  quality  and  compare  alternative  sets  of  requirements 
and  solution  arch Itectures.  Provide  an  assoc lated  methodology 
that  helps  focus  requirements  engineering  activities  to  produce 
better  estimates  of  system  time,  cost,  and  performance. 

Solution  approach:  Develop  a  Metric  Guided  Methodology  with: 

(1)  tools  for  quality  assessment,  ( 2 )  knowledge-based  tools  for 
acquiring  and  manipulating  knowledge  of  the  system  being 
developed,  (3)  support  and  correlate  different  design 
representat Ion  schemes,  and  (4)  combine  predictive  metrics  with 
protctyp i ng . 

Recommendation:  Funding  priority:  Medium.  Risk:  Moderate. 

There  will  be  difficulties  in  validating  metrics  and  In  the 
knowledge-based  aspects  of  the  work,  but  If  these  difficulties  can 
be  overcome,  the  payoff  will  be  high. 


TESTBED  EFFECTIVENESS 

Where  It  fits  In  process  model:  It  relates  to  the  engineering 
context  description. 

Research  objectives:  Provide  the  capability  to  determine  the 
effectiveness  of  new  Requirements  Engineering  Testbed  tools  and 
techniques. 

Solution  Approach:  Instrument  the  RET  for  time/effort  and 
resource  utilization  measurements.  Provide  basis  (metrics, 
prototyping)  for  development  cost/rlsk/schedule  estimation. 

Compare  requirements  quality  before/after  application  of  a  tool  or 
techn ique. 


Recormendat  i  on  :  Funding  priority:  Medium-low.  Mostly  a 
development  concern.  This  effort  might  be  Incorporated  in  the 
"Evolutionary  Testbed  Integration"  development  issue. 


Characterization  of  Evolutionary  Track  Development,  issues 

There  are  three  development  Issues:  User  Interface,  Database,  and 
Evolutionary  Testbed  Integration.  All  three  Issues  form  part  of 
the  panel's  strategy  for:  (1)  Integrating  the  RET,  and  (2) 
Instrumenting  the  RET  as  a  basis  for  tool  evaluation.  The  "User 
Interface"  and  "Database"  issues  also  provide  Initial  versions  of 
correspond  I ng  elements  In  the  long-range  RET  architecture. 

The  development  issues  are  generally  given  the  same  kind  of 
character izat ions  as  were  the  research  Issues,  except  that  they 
are  given  a  character izat ion  of  how  they  fit  in  the  long-range  RET 
architecture  rather  than  cf  how  they  fit  in  the  process  model. 


USER  INTERFACE 

How  it  fits  in  RET  a-ch  I  tecture:  The  use  of  consistent  in  +  er  feces 
by  all  tools  eases  user  access  to  tool  and  data.  Supports  tight 
integration  of  tools. 

Development  objectives:  All  too!  developers  should  be  required  to 
use  a  consistent  approach  to  end-user  communication. 

Solution  Approach:  (1)  Establish  user  interface  models  and 
standards  to  be  observed  in  the  development  of  ell  tool  interfaces 
and  in  the  use  of  run  time  support  packages.  (2)  Check  compliance 
with  standards  by  all  tool  contractors.  (3)  Evolve  standards. 

Recommendat ion:  This  effort  should  be  initiated  very  early  In  the 
R&D  program  so  as  to  affect  all  development  activities. 


DATABASE 

How  It.  fits  In  RET  architecture:  Provide  data  storage  and  data 
management  capabilities  to  support  tight  Integration  of  RET  tools 
via  shared  data. 

Development  objectives:  Provide  viewing/reporting,  editing, 
classification,  and  export/import  facilities  to  aid  RET  users  In 
accessing  and  managing  large  volumes  of  complex  data. 


Sol  union  Approach:  Select  a  general-purpose  DBMS  which  Is 
efficient  in  the  storage  of  design  objects  and  which  provides  the 
proper  facilities.  Develop  common  data  object  descriptions  and 
conversion  routines  so  existing  tools  can  share  data.  Customize 
data  management  facilities  for  RET  needs. 


Recommended i cn :  This  is  a  high-payoff  low-risk  component  cf  the 
RET,  and  critical  to  the  evolutionary  track. 


EVOLUTIONARY  TESTBED  INTEGRATION 

How  It  fits  In  RET  architecture:  This  effort  would  support  tight 
Integration  of  existing  tools,  allowing  the  use  of  key 
capabilities  of  each  tool  on  the  same  set  of  requirements. 

Deve I opment.  ob.J ectJ.v.es :  Develop  an  Integrated  RET  featuring:  an 
RET  experimentation  methodology,  a  common  database  and  user 
Interface  for  tools,  and  a  way  of  bringing  new  tools  In. 

So  I ut  ion  Approach :  Identify  the  degree  of  tool  Interaction  and 
tracking  desired.  Assuming  the  "database"  and  "user  Interface" 
efforts  have  progressed,  begin  modifying  the  tools.  For 
tightening  the  Integration,  Identify  which  tools  should  be  Invoked 
by  data  changes  and  which  should  be  explicitly  invoked  by  the 
user.  Develop  an  RET  experimentation  methodology. 

Recomnendaticn:  This  effort  is  critical  to  achieving  tool 
Interaction  and  also  for  RET  effectiveness  assessment.  Possibly 
incorporate  instrumentation  capabilities  from  "Testbed 
effectiveness"  into  this  effort. 
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As  explained  In  section  4.2.1,  the  theme  of  the  Formal  Language 
Track  is  to  provide  a  formal  treatment  of  requirements.  In 
section  4. 2. 3.1,  we  Indicate,  by  an  R&D  roadmap,  the  panel's 
strategy  of  realizing  this  theme.  This  roadmap  also  shows 
dependencies  between  all  R&D  Issues  of  the  track. 

In  section  4.2.3 .2,  the  research  issues  which  constitute  the 
Formal  Language  Track  are  characterized. 


4.2.3. 1  Objectives  and.  Roadmap 

The  goal  of  the  formal  language  track  is  to  provide  a  formal 
treatment  for  requirements  in  which  automated  support  can  be  given 
for  tracking  requirements  Into  specifications  and  checking  for 
requirements  satisfaction  when  a  specification  Is  run  against 
scenarios. 

The  addressed  parts  of  the  1995  scenario  include: 

All  parts  are  addressed:  formal  support  will  be  given  to 
all  illustrated  synthesis  activities  and  analysis  activities 
except  for  synthesis  of  goals,  for  which  a  methodology  will 
be  provided. 

Figure  4.2.3. 1-1  indicates  the  panel's  strategy  for  providing  a 
formal  treatment  for  requirements.  Each  box  represents  a  research 
issue  of  the  Formal  Language  Track.  (Because  of  the  close 
correspondence  between  Formal  Language  Track  K&D  efforts  and  R&D 
Issues,  there  is  no  need  in  the  roadmap  for  a  special  linkage  such 
as  that  employed  in  the  roadmaps  of  section  4. 2. 2.1.) 
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4. 2. 3. 2  Research  Issues 


The  Formal  Language  Track  consists  only  of  the  research  Issues 
indicated  in  figure  4. 2. 3. 1-1.  Appendix  E  gives  detailed 
characterizations  of  these  Issues.  These  issues  can  be  considered 
as  belonging  to  one  of  two  types:  (1)  Issues  that  directly 
contribute  to  and/or  are  strongly  dependent  on  a  formal 
requirements  language,  and  (2)  Issues  that  relate  to  the  broader 
requirements  engineering  context.  The  latter  research  issues 
assume  the  existence  of  a  formal  requirements  language,  but  It  Is 
assumed  that  their  Investigation  can  be  done  fairly  Independently. 
Below,  we  treat  all  Issues  strongly  dependent  on  the  language  as 
the  "Formal  Requirements  Language"  issue.  All  other  Issues  are 
given  their  own  summaries.  The  style  of  characterization  is 
similar  to  that  used  In  section  4. 2. 2. 2.  Finally,  we  Identify 
candidate  Issues  that  were  rejected  for  inclusion  in  the  Formal 
Language  Track. 
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FORMAL  REQUIREMENTS  LANGUAGE 


Where  Issue  fits  In  process  model;  This  Issue  will  provide  a 
common  formal  language  for  goals,  requirements,  and  solution 
arch itectures.  Thus  analysis  on  each  of  these  and 
synthesis/analysis  between  each  of  these  can  be  given  formal 
automated  support.  This  would  address  most  "transform"  and 
"analysis"  activities  of  the  process  model. 


Research  Object  Ives;  Identified  as  Individual  research  Issues  In 
f Igure  4.2.3. 1-1 :  requirements  integrated  Into  specification 
language,  formal  Interpretation  of  requirements  against  behavior, 
goal  coverage  analysis,  multiple  levels  of  abstraction,  and  an 
incremental  requirements  language. 


Solution  Approach;  Expand  existing  formal  specification  language 
to  Include  formal  requirements  statements.  Share  a  common  domain 
model  and  define  requirements  as  predicates  against  behavior  of 
specification.  Formally  execute  specification  to  generate 
behavior  against  which  to  test  requirements  predicates.  Include 
goals  as  requirements  which  can  be  further  refined.  Provide 
support  for  multiple  levels  of  abstraction  in  stating  requirements 
and  specifications  and  mapping  between  them.  Provide  support  for 
evolving  requirements  statements  on  basis  of  feedback  from 
evaluation  tools. 


Recommendation:  Funding  priority:  High.  Risk:  low  for 
integrated  language;  high  for  reasoning  and  analysis  tools. 

Formal  language  approach  to  requirements  Is  highly  recommended  as 
a  complement  to  the  Evolutionary  Track.  It  Is  higher  risk  and 
higher  payoff  and  that  payoff  occurs  later  than  In  the 
Evolutionary  Track.  But  It  lays  the  foundation  for  earlier  and 
more  reliable  detection  of  requirements  problems  and  their  use  as 
a  real  design  envelope. 


METHODOLOGY  FOR  FORMAL  REQUIREMENTS  SYNTHESIS 


and  requirements. 


Guides  the  synthesis  of  goals 


Research  Ob j ect I ves :  Provide  guidance  for  mission  users  and 
acquisition  engineers  In  creating  formal  requirements. 


Solution  Approach:  Extend  structured  specification  methodologies 
to  requirements  and  their  formal  expression.  Extend  the 
methodology  to  expression  of  goals  and  determining  how  to  revise 
and  ref Ine  them. 


Recommendation:  Funding  priority:  Medium-high.  Risk:  moderate 
to  high.  This  research  offers  much  promise  in  that  it  aids 
getting  conceptualizations  Into  formalisms,  thereby  making  Formal 
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Language  Track  facilities  accessible  to  a  broader  range  of  users, 
but  success  Is  uncertain. 


SCENARIO  GENERATION  AND  COVERAGE 

Where  Issue  fits  In  process  model:  In  rapid  prototyping: 
building  scenarios  that  drive  prototypes  and  by  examining  the 
generated  behavior,  determining  whether  the  requirements  are 
correct  and  complete.  Requirements  evaluation  and  reformulation: 
Issue  also  addresses  support  for  determining  whether  the 
requirements  are  satisfied  under  a  class  of  scenarios. 

Research  Objectives:  Provide  tool  which  determines  whether 
requirements  are  satisfied  by  a  specification  with  respect  to  a 
particular  scenario.  (The  value  of  this  check  Is,  assuming 
satisfaction,  any  undeslred  behavior  detected  by  the  user  during 
execution  of  the  specification  against  the  scenario  will  Imply 
Incorrect  or  incomplete  requ I rements. )  Expand  this  tool  to  handle 
classes  of  scenarios.  Support  automatic  generation  of  a  complete 
scenario  from  a  mission  user's  outline  of  desired  behavior. 

Solution  Approach:  For  generation  of  scenarios:  reverse  the  flow 
of  reasoning  In  symbolic  evaluation  to  deduce  the  class  of  Inputs 
to  a  specification  that  generates  the  behaviors  desired  by  the 
user.  For  checking  satisfaction  of  requirements  against 
specification  (prototype)  behavior:  use  abstraction 
mapping/matching  mechanisms  to  check  whether  the  behavior  the 
prototype  generated  is  a  legal  instantiation  (legal  relative  to 
the  requirements). 

Recommendation:  Funding  priority:  High.  Risk:  near-term:  low 
to  moderate,  but  moderate  to  high  for  handling  classes  of 
scenarios.  Automatic  generation  of  scenarios  and  checking  for 
requirements  satisfaction  are  among  the  highest  leverage 
capabilities  of  either  track. 


MANAGING  RESOURCES 

Where.  .Issue  fits  In  process  model:  Supports  all  requirements 
engineering  activities  by  managing  the  human  and  computing 
resources.  Provides  a  basis  for  effectiveness  measurements. 

Research  Objectives:  Manage  the  human  and  computing  resources 
needed  to  engineer  a  set  of  requirements,  and  track  the  resources 
through  the  engineering  process. 

Sol ution  Approach:  Formally  describe  all  requirements  engineering 
activities  as  tasks;  Identifying  their  dependencies,  resources 
consumed,  and  results  produced.  Construct  task  manager  which 
understands  these  descriptions  and  guides  user  In  task  selection. 
For  multi-user  efforts,  expand  task  manager  to  coordinate  all 
tasks  and  use  of  tools  among  users. 


Recommendation :  Funding  priority:  Medium.  Risk:  moderate  to 
high.  Success  In  this  issue  Is  strongly  dependent  on  success  In 
most  other  Formal  Language  Track  Issues. 


Rejected  Issues 

The  following  issues  were  rejected  for  Inclusion  Into  the  Formal 
Language  Track.  Reasons  for  the  rejection  are  given. 


REQUIREMENTS  ANALYSIS  METHOOOLOGY 

Rejected  because  existence  of  a  formal  language  Implies  a 
basis  for  the  formal  treatment  of  descriptions,  with  "tools" 
automatically  Invoked  In  a  data-driven  problem-dependent 
manner.  The  "coordination"  of  such  Invocations  Is  left  to 
be  addressed  in  the  "Managing  Resources"  research  issue. 

SCENARIOS  EXECUTION  AND  ANALYSIS  OF  RESULTS 

Rejected  as  a  separate  effort  because  of  its  very  strong 
dependence  on  the  formal  language.  However,  rapid 
prototyping  will  provide  high  leverage  given  the  formal 
language  approach;  with  additional  value  in  the  formal 
verification  possible  between  the  prototype  and  requirements 
(with  respect  to  particular  scenarios).  This  capability 
will  be  included  in  other  Formal  Language  Track  efforts. 

NATURAL  LANGUAGE 

Considered  as  an  aid  to  knowledge  acquisition  and 
formalization  of  descriptions.  Rejected  because  that 
technology  is  already  being  extensively  explored  and  Its 
development  Is  Independent  of  our  Intended  requirements  use. 
As  It  matures,  It  will  undoubtedly  be  Incorporated  Into  the 
RET.  It  was  rejected  as  a  RET-speciflc  research  area. 

SOLUTION  ARCHITECTURE  SYNTHESIS 

Rejected  because  that  technology  Is  being  extensively 
explored  and  It  examines  an  Issue  orthogonal  to  the  primary 
focus  here.  However  It  will  provide  high  leverage  as  It 
provides  a  basis  for  generation  of  rapid  prototypes.  As 
the  technology  matures,  ft  will  be  evaluated  for 
Incorporation  Into  the  RET. 
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This  section  presents  panel  recommendations  on  the  priorities  of 
issues  within  each  track  and  on  the  recommended  allocation  of  R&D 
resources  between  the  two  tracks. 


4.2.4. 1  Prioritization  in  the  Evolutionary  Track 

Research  Issues  In  the  Evolutionary  Track  were  prioritized  In  the 
order  of  preferred  funding  by  the  Requirements  panel.  Development 
Issues  were  not  prioritized  relative  to  each  other  as  they  were 
all  regarded  as  critical  to  the  Tool  Evaluation  activity  (see 
figure  4. 2. 2. 1-3).  The  prior Itizatlon  of  research  issues  follows: 


HIGH: 

Scenarios  Execution  and  Analysis  of  Results 

Estimation  of  Cost,  Risk,  Time  In  System  Development, 
Performance  &  Execution  Costs  Analysis 

Scenario  Generation  Support 

&  Scenario  Coverage  Analysis 

MEDIUM-HIGH: 

Domain  Models  and  Information 
Dynamic  Analysis 

Requirements  Analysis  Methodology 
MEDIUM: 

Requirements  -  Static  Analysis 
Requirements  Evaluation 
MEDIUM-LOW: 

Goals  and  Requirements  Synthesis 
Testbed  Effectiveness 


LOW: 

Validation  of  Prototype  and  Scenarios 
Solution  Architecture  Synthesis 
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4. 2. 4. 2  Prior ftlzatlon  In.  the  Formal  Language  Track 

Research  issues  In  the  Formal  Language  Track  were  prioritized  In 
the  order  of  preferred  funding  by  the  Requirements  panel.  As 
explained  In  section  4. 2. 3. 2,  all  language-dependent  Issues  were 
treated  as  a  single  Issue,  the  "Formal  Requirements  Language" 
Issue.  The  prioritization  of  research  Issues  follows: 


HIGH:  3 

Formal  Requirements  Language  J 

Scenario  Generation  and  Coverage 

MEDIUM-HIGH: 

Methodology  for  Formal  Requirements  Synthesis 
MEDIUM: 

Managing  Resources 


4 . 2 . 4 . 3  Allocation  of  Resources  to  Both  Tracks 

To  establish  a  relative  allocation  of  R&D  resources  between  the 
tracks,  the  panel  partitioned  the  R&D  Issues  Into  four  groups  and 
prioritized  them.  The  groups  were:  Evolutionary  Track  research 
issues.  Evolutionary  Track  development  Issues,  the  Formal 
Requirements  Language  Issue,  and  Formal  Language  Track  other 
issues  (see  section  4. 2. 3. 2  for  explanation). 

The  result  of  the  allocation: 


Group: 

Evolutionary  Track  research  issues 
Evolutionary  Track  development  Issues 
Formal  Requirements  Language  Issue 
Formal  Language  Track  other  Issues 


Funding  al location: 
30$ 

29 $ 

23$ 

18$ 


To  summarize,  the  panel  suggests  allocating  funds  between  the 
Evolutionary  Track  and  Formal  Language  Track  on  a  60$/40$  basis. 
Within  the  Evolutionary  Track,  equal  weight  Is  given  to  the 
research  and  development  Issues.  Within  the  Formal  Language 
Track,  the  funds  should  be  allocated  between  the  language  and 
other  Issues  on  a  60$/40$  basis. 
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5.0  CONCLUSIONS 

J 

The  panel  recommends  that  RADC  pursue  a  Requirements  Engineering  ' 

Testbed  (RET)  research  and  development  program  consisting  of  two  ij 

tracks:  an  Evolutionary  Track  and  a  Formal  Language  Track.  Their  j 

strategies  are  summarized  below.  j 

The  Evolutionary  Track  I 

The  "Evolutionary  Track"  proposes  an  evolutionary  R&D  effort  to  \ 

extend  the  current  formalisms  and  tools.  Initial  efforts  are  1 

toward  the  development  of  tools  for  prototyping  Interfaces  and  ! 

functional  tty,  and  In  deriving  performance  estimates  based  on 
estimated  or  simulated  work  loads.  Future  efforts  would  develop  ; 

tools  and  methods  that  aid  In:  (1)  scenario  development, 

analysis,  and  execution,  (2)  cost,  risk,  and  performance  analysis,  ‘ 

(3)  the  acquisition,  modeling,  and  usage  of  domain  Information, 
and  (4)  requirements  analysis  methodology.  Future  efforts  will  j 

also  go  toward  enhancing  the  prototyping  tools  and  formalisms.  " 

The  Evolutionary  Track  also  proposes  several  development  efforts. 

(1)  A  database  is  needed  to  manage  complex  data  such  as 

requirements,  prototypes,  etc.,  and  would  feature  powerful  viewing  ■ 

mechanisms  to  simplify  presentation  of  data.  The  database  would 

also  serve  as  the  central  repository  for  data  and  permit  sharing  1 

data  between  tools.  (2)  A  reconf Igurab I e  user  Interface  Is  I 

needed,  and  must  provide  uniform  access  to  all  tools  and  data  j 

objects.  (3)  Testbed  tools  must  be  tightly  Integrated,  permitting  J 

their  functionality  to  be  manually  Invoked  or  Invoked  with  changes 

In  data  (data-driven  control).  The  Integrated  testbed  must  also 

track  RET  tool  use  to  support  later  analysis  of  tool  1 

effectiveness,  and  guide  the  user  in  tool  usage  (e.g.  j 

methodolog ies) . 

Ihe.. formal.. .Lana.u.eae  Track 

Currently,  requirements,  specifications  and  prototypes  are 
partially  (or  completely)  Informal,  and  are  separately  and 
manually  produced.  This  Informality  precludes  tools  which  compare 
one  level  with  the  next  and  limits  the  types  of  analyses  that  can 
be  done  within  a  single  level.  The  "Formal  Language  Track" 
attempls  to  eliminate  these  difficulties  by  creating  a  common 
formalism  In  which  all  three  levels  are  expressed  and  In  which  the 
prototype  can  be  generated  automatically  from  the  specification. 

Tools  would  be  provided  for  analyzing  the  requirements  and 
specifications  and  formally  comparing  them.  The  common  formal 
basis  for  requirements  and  specifications  would  be  used  to 
automate  the  generation  of  scenarios  to  determine  the  satisfaction 
cf  requirements  In  a  specification  and  the  completeness  and 
appropriateness  of  the  requirements  themselves. 

Thus,  the  Formal  Language  Track  proposes  research  effort  be  spent 
toward  developing  a  single  formal  language  for  expression  of 
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goals,  requirements,  and  solution  architectures  (process  model 
terminology).  Additional  effort  would  go  toward  researching 
relevant  Issues:  (1)  scenario  generation  and  analysis,  (2) 
synthesis  methodologies  (getting  conceptualizations  Into  formal 
descriptions),  and  (3)  the  management  of  resources  (coordinating 
use  of  the  resources  supporting  formal  analysis). 

Contrasting  the  Two  Tracks 

To  contrast  the  two  approaches,  the  formal  language  approach 
offers  much  more  formal  analysis  to  be  done  and  thus  has  high 
payoff,  but  requires  considerable  research  before  It  will  be 
available.  The  evolutionary  approach  permits  earlier  RET  hosting 
of  capabilities  such  as:  prototyping,  scenario  development  and 
execution,  and  domain  information  collection  and  usage.  The 
consequence  is  that  Air  Force  users  will  be  able  to  exercise  these 
capabilities  on  their  requirements  problems  before  availability  of 
the  formal  language.  Also,  the  risk  is  distributed.  These 
complementary  risk/reward  profiles  strengthen  the  program  and 
provide  a  natural  phasing  of  capabilities. 

Though  not  investigated  In  any  detail,  it  is  expected  that  some 
positive  results  developed  In  one  track  may  impact  or  eliminate 
approaches  being  tried  in  the  other,  to  take  immediate  advantage 
of  the  result. 

To  the  extent  that  formalisms  are  adopted  in  the  Evolutionary 
Track,  the  distinctions  between  the  two  tracks  will  be  reduced  or 
el iminated. 


RADC's  currently-contracted  tools  will  be  hosted  in  the  RET  by 
early  1988.  These  tools  will  initially  be  loosely  coupled,  but  by 
1990,  the  objective  is  to  tightly  integrate  them,  and  other  new 
tools,  through  the  RET  database. 


A  few  years  later,  capabilities  from  the  Formal  Language  Track, 
beginning  with  a  common  requirements  and  specification  language, 
will  be  available  In  the  RET  for  test  and  evaluation.  Thereafter, 
both  tracks  will  be  strengthening  their  tools  and  methods  and 
using  the  RET  as  a  test  and  evaluation  vehicle. 
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6.0  EPILOGUE:  RESULTS  FROM  A  REVIEW  OF  THIS  REPORT 
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Following  completion  of  this  report,  copies  were  distributed  for 
review.  This  section  presents  reviewers'  comments  and 
suggestions. 


The  review  was  carried  out  by  computer  scientists  from  both 
academia  and  Industry.  Their  comments  and  suggestions  were 
solicited  on  the  first  five  sections  of  the  report.  The  reviewers 
were: 


Mack  W.  Alford  -  General  Electric,  Valley  Forge, 
Pennsy I  van  I  a. 


MCC,  two  anonymous  reviewers  selected  by  Laszlo  A.  Belady  - 
MCC,  Austin,  TX, 


Harlan  Mills  -  IBM  and  University  of  Maryland,  Maryland, 


Gru la-Catal in  Roman  -  Washington  University,  St.  Louis, 
M I  ssour  i . 


The  comments  below  are  reported  anonymously.  No  comment  should  be 
taken  as  the  unaminous  opinion  of  all  the  reviewers,  in  fact  there 
was  no  comment  common  to  all  reviewers. 


Summarizing,  the  report  was  said  to  be  technically  strong,  though 
with  shortcomings. 


In  general,  the  recommendations  on  the  Requirements  Engineering 
Testbed  Research  and  Development  Program  were  perceived  as  sound, 
going  a  good  way  toward  addressing  problems  later  encountered  In 
systems  development. 


The  Requirements  Engineering  Testbed  concept  as  a  vehicle  for 
technology  transfer  and  for  maintaining  research  In  systems 
definition  was  also  considered  to  be  sound. 


In  the  minds  of  some  reviewers,  the  most  serious  shortcomings 


(I)  The  recommendations  lean  too  heavily  on  formal 
specification  techniques. 


(2)  The  recommendations  do  not  rest  on  a  sufficiently 

adequate  foundational  basis,  affecting  the  credibility 
of  some  of  the  recommendations  on  tools. 


Responding  to  the  first  comment.  It  is  the  consensus  of  the  panel 
that  formal  specifications  will  have  a  big  payoff  in  the  long 


s  s  s  . 


term,  but  relying  on  then  in  the  short  term  Is  naive.  That  is  why 
at  least  two-thirds  of  the  1990  Requirements  Engineering  Testbed 
is  based  on  "informal"  approaches,  l.e.  the  Analyst  and  RPS.  The 
role  of  formal  specifications  in  the  testbed  is  expected  to 
gradually  become  more  prominent  after  that. 

Responding  to  the  second  comment,  the  panel  agrees  on  the  absence 
of  a  strong  foundational  basis,  and  knows  of  none.  One  major 
reason  why  long-term  research  and  development  programs  are  funded 
is  to  provide  opportunities  to  Identify  such  a  foundational  basis. 
One  way  to  do  this  Is  to  Identify  and  develop  promising  Interim 
approaches  that  give  rise  to  new  Issues  and  Ideas  that  can  be 
explored.  In  the  report,  the  panel  attempted  to  define  such  a 
long-range  program. 

Additional  Opportunities 

All  reviewers  identified  one  or  more  research  and  development 
opportunities  they  felt  should  be  investigated: 

(1)  Simple  solutions  such  as  enhancing  the  communication 
between  mission  users  and  system  engineers,  and  using 
good  people  to  do  the  systems  engineering. 

(2)  hhat  to  do  on  ill-defined  problems. 

If  the  problem  can't  be  readily  defined  beforehand, 
all  of  the  techniques  explored  by  the  panel  provide 
I ittle  help. 

(3)  The  "Requirements  Volatility  Problem".  Mission  and 
user  needs  change  with  time,  thus  the  requirements 
change  with  time. 

(4)  A  mere  flexible  representation  for  scenarios  and 
scenario  sets.  Benefits  obtained  would  Include  a 
project-wide  glossary,  and  the  use  of  scenarios  to 
explore  the  boundaries  of  system  capabilities. 

Responding  to  the  first  half  of  (1),  and  acknowledging  (3),  It 
should  be  recognized  that  the  panel  was  Investigating  the 
requirements  problem  In  the  context  of  government  procurement, 
which  makes  some  solutions  to  these  Issues  Impractical. 

Responding  to  the  second  half  of  (1),  the  panel  agrees  on  the 
Importance  of  using  good  people  to  do  the  systems  engineering. 

Responding  to  (2),  the  panel  voiced  no  technique  for  exploring 
ill-defined  problems  other  than  prototyping,  evaluating  the 
prototype  in  near-actual  use,  and  Iterating.  The  panel  feels  that 
the  requirements  process  model  already  addresses  this,  though 
imp  I ic it  I y . 


Responding  to  (4),  the  report  recommends  that  "Scenario  Generation 
Support  &  Scenario  Coverage  Analysis"  and  "Scenario  Execution  and 


Analysis  of  Results”  research  and  development  Issues  receive  high 
priority  for  funding.  In  short,  the  panel  agrees  with  the 
reviewer  on  the  Importance  of  scenarios,  though  maybe  feels 
differently  on  where  to  place  emphasis. 


One  reviewer  recommended  that  a  critical  objective  of  the 
Requirements  Engineering  Testbed  should  be  to  close  the  gap 
between  ”goals”,  formulated  In  terms  of  mission  concepts,  and 
"requirements",  formulated  in  terms  of  system  concepts. 

The  panel  feels  this  to  be  a  presentation  Issue.  It  feels  the 
Issue  is  addressed  In  the  proposed  program,  and  was  one  major 
reason  for  making  the  distinction  between  "goals"  and 
"requirements"  In  the  requirements  engineering  process  model. 

In  addition  there  were  a  number  of  comments  recommending  further 
thought  on  parts  of  the  process  model:  addressing  performance 
requirements,  design  decomposition  and  representation,  and  the 
affect  of  size  and  complexity  on  the  requirements  problem. 

A  serious  concern  was  expressed  on  the  tool  developments  implied 
by  the  1995  scenario.  The  reviewer  felT  that  mere  thought  was 
required  on  their  justification. 

The  panel  agrees  on  the  conjectural  aspects  of  the  1995  scenario. 
The  panel  was  attempting  to  project  feasible  capabilities,  as  a 
means  of  identifying  long-range  objectives  in  the  Formal  Language 
Track  of  the  research  and  development  program.  As  knowledge  is 
acquired  through  the  Requirements  Engineering  Testbed  program,  the 
direction  of  anticipated  tool  developments  should  be  revised. 

Summary 

Summarizing,  the  major  concerns  seem  to  be: 

*  A  better  foundation  for  representing  requirements  and 
techniques  for  eliciting  and  expressing  them. 

*  Continued  reevaluation  of  the  formal  specifications 
tool  development  program  as  to  direction  and 

justi f Ication. 

*  A  more  flexible  contracting  process,  permitting  a  more 
opportunistic  Interleaving  of  definition,  development, 
and  evaluation  efforts.  Also,  permitting  better 
communication  between  mission  users  and  systems 

eng  I neers. 


APPENDIX  A:  DETAILED  CHARACTERIZATIONS  OF  TESTBED  USERS 


This  section  provides  detailed  characterizations  of  three 
classes  of  Testbed  users:  Mission  users.  Acquisition  engineers, 
and  Software  developers. 

Characterizations  are  given  for  three  different  times:  "Now", 
"5  Years",  and  "10  Years".  For  each  user  class  and  each  time,  the 
following  Is  described: 

*  BACKGROUND  -  degree  of  user  sophistication  with  computer 

tools,  languages,  and  applications 

*  ISSUES,  QUESTIONS  -  what  are  the  user's  concerns  with  the 

requirements,  what  Is  his  role? 

*  HELP  -  What  kind  of  support  would  enable  the  user  to 

satisfy  his  objectives? 

*  TECHNICAL  ISSUES  -  What  technical  Issues  must  be 

addressed  If  help  Is  to  be  given  to  the 

user? 

As  RET  R&D  program  focus  Is  on  helping  the  Mission  user  and 
Acquisition  engineer,  "Help"  and  "Technical  issues"  descriptions 
are  net  provided  for  the  Software  developer. 

For  each  testbed  user;  "Issues,  questions",  "Help",  or 
"Technical  Issues"  character Izat Ions  that  appear  early  in  the  time 
line  generally  persist  throughout  the  remainder  of  the  time  line. 

Tc  conserve  space,  we  avoid  repeating  them. 
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Mission  User. 


ULfflfi: 


*  Access  to  "time-shared" 
text-oriented 


*  Computer  I  Iterate,  but  no 
computer  specializations 


*  (Typically)  manages  message 
hand  I Ing  systems  : 

accessing/correlating  info 
from  local  and  remote 
databases,  notifying  others 


*  Limited  networking: 

(1)  user-to-user  on  mission- 
related  job 

(2)  transmittal  of  requirements 
documents  among  among  acquisition 
eng Ineers 


Time: 


5  years 


*  Graphical  interface  (vs.  textual): 
forms/ icons  style  of  computer  dialog 


*  One  PC  per  desk 


*  Networking  standard  among: 

(1)  user-to-user  with  respect  to 
a  mission-related  job; 

(2)  (RAOC)  acquisition  engineers, 
sending  back  and  forth 
requirements  documents 


*  Strong  resistance  to  formal  methods 
(prefers  learning  time  of  1  hour) 


•Will  I  be  able  to  accompl Ish 
terminals  my  mission? 


•Will  I  survive  under  this 
scenario? 


•Will  I  be  "frozen  out"  because 
the  computer  system  has  removed 
control  from  me? 


*  Are  accuracy  and  response  time 
suf f  Icient? 


*  Is  the  interface  oriented  to 
my  needs? 


*  As  the  system  wears  down 
through  attrition,  will  I  be 
able  to  Invoke  appropriate 
back-up  systems? 


•  What  diagnostics  are 
automat ic? 


*  As  the  system  wears  down 
through  attrition,  will  It 
degrade  "gracefully"? 


♦  What  repairs  are  automatic? 


tions  relating  to 
ir  the  requirements 
ght: ) 


ey  fulfill  the 
s  needs? 


he  5ytem  be  built? 


*  Very  limited  office  automation  (10$)  *  Are  mission  user  concerns/needs 

understood  by  the  developer? 

*  Is  the  mission  user  sensitive  to 
development  constraints? 

*  Is  each  requirement  quant  I f  lab  I y 
demonstrab I e? 

*  Can  procurement  selection  be 
Justified  on  technical  grounds? 

*  Is  competition  being  fostered 
(will  there  be  many  responses  to 
my  RFP)? 


Time:  5  Years 

background : 

*  One  PC  per  desk 

*  Limited  networking: 

(1 )  with  mission  user, 

(2)  possible  objective:  tie 
developer  in  for  big  development 
programs 

*  Limited  office  automation  (20$) 

*  Increased  specializations 

*  Limited  prototyping  for  requirements 
evaluation  and  validation  (associated 
with  Incremental  development, 
especially  Interfaces) 

*  Metrics  for  evaluating  alternatives 


lime:  10  Years 

background :  Issues..  .Questions: 

*  LANs,  providing  access  to  a  wide  *  Can  software  procurements  go 

spectrum  of  computer  resources  fixed  price,  warrantled? 

*  Limited  decision  tools  (little  AI) 

*  Breadth  of  computer  knowledge 


issues,  questions: 

*  Which  requirements  should  be 
evaluated? 

*  For  certain  requirements,  are 
there  alternatives?  What 
costs/benefits  do  they  entail? 


*  Office  automation  (50$) 


*  Al-based  decision  tools 

*  wide  use  of  prototyping  for 
requirements  evaluation  and 
val Idatlon 


Software  Developer. 


Time:  Now 

background: 

*  One  language  fluency 

*  Workstations  on  LANs 

*  Access  to  large  machine 

*  Office  automation  support 

*  No  automated  allocation  to  multiple 
processors 

*  Large  group  of  helpful  peers 

Time:  5  Years 

b.ac.kflrfluml: 

*  Highly  supportive  workstation 

*  Fluency  wtth  several  workstation 
environments 

*  One  physical  workstation  per 
desk,  on  networks 

*  Comfortable  in  using  a  number  of 
1 anguages 

*  No  des Ign/arch itecture  support  tools 

*  Development  host  and  operational 
host  separation 


issues,  questions: 

*  Do  requirements  over-spec  I fy  a 
design?  Or,  are  requirements  too 
mission-oriented  (high-level)? 

*  Does  the  development  staff  have 
the  requ I  red  skills? 

*  What  are  the  hard  real-time 
problems  (which  functions  in 
which  scenarios  produce 
bottlenecks)? 

*  Are  cost/schedule  constraints 
compatible  with  funct lon/perfoi — 
nance  requirements? 


Issues,  questions: 

*  Can  I  eke  out  a  competitive  edge 
through  systematic  experimen¬ 
tation  of  system  alternatives? 


*  Does  my  development  team  have 
access  to  and  knowledge  of  the 
best  development  tools  and 
techniques  to  do  the  job? 


*  Functions  parsed  Into  multiple 
processes 

*  Breadboarding  of  all  critical 
design  Ideas 


Time:  10  Years 

background:  Is.s.ues,,  .questions: 

*  Whole  environment  fosters  *  How  can  I  demonstrate,  or  prove 

”compet It  I ve  differentiation”  a  priori,  error-free  operation? 

*  Prototyping  testbed  for  evaluating 
real-time  system  requirements 

*  Design/architecture  concept  support 
tool  s 

*  Expert  system  technology  matures 

*  Build  software  first,  then  order 
hardware  highly  tailored  to 
software  architecture 


Now  a  characterization  of  the  kinds  of  support  that  would  enable  the  mission 
user  to  attain  his  objectives. 


Mission  User. 


Time:  Now 

help; 

*  Expert  committee  pul  led  together 
by  mission  user 

*  Limited  Incremental  development 
(pre-planned  program  Improvement) 
for  putting  together  large  systems 


Time:  5  Years 

help: 

*  Conceptual  model  (mission-oriented, 
stated  In  English)  of  what  Is  needed 
(basis  of  statement  of  need) 

*  Limited  use  of  operational  models 
of  major  subsystems;  providing 
opportunities  to  stress-test  criti¬ 
cal  functions 


*  Some  isolated  expert  systems 
based  upon  domain  models 


technical  Issues: 

(partial  solutions  already 
exist) 


*  Application-oriented  VHLL  for 
prototyping  and  simulation 


*  Advanced  d3ta  base  concepts 

*  Real  time:  user  experiences 
unfolding  battle 

*  Reusab i I ity 

*  To  support  real-time  simulation: 
high-speed  processors 

*  Risk  evaluation  via  rapid 
prototyping  (seeding  errors  at 
requirements  level) 

*  C3I  rapidly  reconf Igurab le 
environmental  simulator  (hardware 
and  VHLL) 

*  Parallel  algorithms  to  provide 
opportunities  for  concurrent 
process  I ng 


Time: 


10  Years 


hgi£: 

*  Operational  models  widely  used 

*  Partially  integrated  domain  models 


*  Artificial  Intelligence: 
**  Rule  generator  for  VHLL 
**  Rule  cr Itiquer 
**  domain-model  support 


A  characterization  of  the  kinds  of  support  that  would  enable  the  acquisition 
engineer  to  attain  his  objectives. 


Acquisition  Engineer. 


help: 

*  Very  limited  rapid  prototyping 


(partial  solutions  already 
exist) 


*  Incremental  development  routinely 
done 

*  Simulation  models 

*  Scenario  generation  (via  graphics, 
etc. ) 

*  Cost  estimation  models 

*  Proposal  evaluation:  arch itecture 

eva I uat 1  on  espec 1  a  I  I v  (i.e.  high  level 
design)  (mental  exercise);  program 
development  plan  (mental  exercise) 

*  Formal  review  process  (no  separation 
of  views),  and  walkthroughs 

*  Experimental  use  of  new  languages, 
tools,  methodologies  (especially  In 
the  validation  of  the  requirements 
delivered  by  the  contractor) 

*  Performance  models  (assessments 
of  computer  configuration 
(through  queueing  models)) 


w>vx  vx/v  l-. 
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lime; 

he  l  p  •• 


5  Years 


*  Limited  use  of  rapid  prototyping 
for  evaluation  and  validation  of 
requ I rements 

*  risk/cost  evaluation  of  alternatives 
(for  requirements  specification  and 
evaluation  of  design  approaches  In 
proposal s) 


*  Static  analysis  of  requirements: 
**  syntax-check i ng 
**  ad-hoc  measures  for  evaluating 


*  Ad-hoc  measures  of  evaluating  the 
tools  that  produced  the  requirements 

*  Viewpoint  enactment  tools  in 
limited  use  (separation  of  views) 


*  common  VHLL/database  for 
exercising  requirements  with 
many  tools 

*  VHLL  for  rapid  prototyping 

*  advanced  data  base  concepts  for 
knowledge  representation  (domain 
model  I ing)  and  update 
dependencies 

*  distributed  environment 


*  test  case  generator  and  scenario 
generator  for  stress-testing 


*  risk/cost  models/metrics 


*  domain  models  (functionality, 
performance,  test  data 
characterization) 

*  (other  Issues  similar  to  mission 
user) 


10  Years 


help: 


*  Fully  Integrated  models: 

##  domains 

**  simulation 
**  system  definition 
*#  scenario 
**  val Idation 

*  Significant  use  of  rapid  prototyping 
for  evaluation  and  validation  of 
requ i rements 

*  Increasingly  formal  and  semantical 
static  analysis 


*  Artificial  Intelligence: 

«*  mission  — >  plans  representation 
#*  model  I Ing  support 
**  update  dependencies  support 

*  Advisor  on  procurement  strategies 


*  * 

I 
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Time; 


15  Years 


help: 

*  Requirements  specification  standards: 
Each  requirement  Is  a  quant! f I  ably 
measurable  augmentation  or  constraint 
to  system  behavior  (through  testing  or 
IEEE  standard  evaluation) 


APPENDIX  B:  DOD  STANDARD  2167  AND  THE  REQUIREMENTS  ENGINEERING  TESTBED 
PROCESS  MODEL 


DoD-S+d-2167  prescribes  requirements  for  the  development  and 
acquisition  of  software  In  terms  of  six  software  development  life 
cycle  phases:  software  requirements  analysis,  preliminary  design, 
detailed  design,  coding  and  unit  testing.  Integrating  and  testing 
of  Computer  Software  Components  (CSCs  are  aggregates  of  software 
units)  and  testing  of  Computer  Software  Configuration  Items  (CSCIs 
are  the  highest  level  aggregation  of  CSCs  and  units).  Its 
emphasis  Is  primarily  managerial.  It  concentrates  on  the  planning 
and  scheduling  of  products  and  reviews  by  which  a  software 
development  and  acquisition  can  be  controlled.  Technical  guidance 
on  how  the  software  should  actually  be  designed  and  built  Is 
minimal.  It  Is  implied  by  the  hierarchical  structuring  of  the 
C3CI,  CSC  and  software  units  to  be  a  top  down  decomposition  based 
on  functional  requirements,  data  flow  requirements  or  other  design 
considerations.  The  software  development  model  prescribed  by  2167 
Is  also  directed  for  use  in  the  development  of  software  in  all 
stages  of  the  system  life  cycle.  This  includes  concept 
exploration,  demonstration  and  validation,  full  scale  development 
and  production  and  deployment. 

The  Requirements  Engineering  Testbed  (RET)  process  model  is  a 
partially  iterative  set  of  transformations  to  a  set  of 
descriptions  which  take  user  needs  from  a  very  informal,  wish  list 
stage,  to  formal  requirements  and  partial  designs.  Its  primary 
intent  Is  In  describing  the  technical  nature  of  the  products  and 
the  transformations  which  they  undergo.  It  does  not  explicitly 
address  management  and  acquisition  issues.  It  is  intended  to 
iteratively  evolve  user  needs  into  requirements  during  the 
software  development  activity  which  precedes  Implementation. 

There  Is  nothing  Inherent  In  the  model  which  would  preclude  its 
use  during  post  implementation  activities,  for  example,  to 
incorporate  changes  to  user  needs  and  their  related  requirements. 

The  purpose  of  this  appendix  Is  to  assess  the  relationship  between 
2167  and  the  RET  process  model.  All  of  the  Software  Requirements 
Analysis  phase  activities,  products,  reviews  and  developmental 
baselines  defined  by  2167  will  be  Identified  and  their 
relationship  to  the  RET  model  will  be  explored  In  terms  of  the 
kind  of  Information  which  the  RET  model  produces  and  which  would 
be  usable  within  the  requirements  of  2167. 

Software  Requirements  Analysis  Activities  defined  by  2167 

Establish  control  of  various  management  documents,  such  as, 
the  Software  Development  Plan  (SDP),  Software  Standards  and 
Procedures  Manual  (SSPM),  Software  Configuration  Management 
Plan  ( SCMP )  and  Software  Quality  Evaluation  Plan  (SQEP). 


None  of  these  plans  or  manuals  are  addressed  by  the  RET 
model . 

Analyze  the  previously  developed  developed,  prel imlnary 
Operational  Concept  Document  (OCD)  and  describe  the  system 
mission.  Its  environment  and  the  functions  of  the  system’s 
computer  system. 

The  OCD  Is  a  description  of  how  the  target  system  will 
appear  to  Its  users  and  the  ways  that  the  users  will 
Interact  with  It.  The  OCD  provides  a  statement  of  the  role 
or  mission  of  the  target  system.  It  describes  the 
physical/tactical  environment  In  which  the  target  system 
will  be  used  and  Identifies  Its  performance  parameters.  It 
lists  and  describes  the  functional  roles  in  which  the  target 
system  will  support  its  users.  It  describes  the 
capabilities  which  it  will  provide  to  these  users  In  these 
roles.  It  outlines  the  hardware  (Including  computer) 
complex  which  will  be  required  to  support  the  target  system. 
Finally,  It  gives  the  plans  for  maintaining,  upgrading  and 
adapting  (to  changing  roqu i rements)  the  target  system. 

The  RET  process  model  recognizes  that  good  user 
understanding  of  the  mission,  application  and  specific  job 
actions  is  critical  in  performing  the  RET  process  model 
transformations  which  take  wishes  to  goals  and  goals  to 
requirements.  This  body  of  knowledge  currently  resides  with 
the  user.  The  RET  model  recognizes  the  need  to  formally 
represent  It  In  domain  models.  This  information  could  be 
used  to  help  directly  in  the  analysis  and  description 
activities  which  2167  requires. 

Analyze  for  adequacy,  testability,  understandab i I ity,  validity  and 
completeness  the  previously  developed  preliminary  versions  of 
technical  specifications,  such  as  the  System/Segment  Specification 
(SSS),  System  Requirements  Specification  (SRS)  and  Interface 
Requirements  Specification  (IRS). 

In  the  context  of  2167  this  activity  and  Its  predecessor, 
l.e.,  analyze  OCD,  form  a  logical  sequence  of  events  leading 
up  to  the  (next  activity)  definition  of  requirements.  The 
RET  process  model  does  not  explicitly  Incorporate  the 
analysis  of  "outside"  requirements  or  specifications.  It 
assumes  that  all  such  Information  will  have  originated 
within  the  RET  model,  will  be  expressed  In  a  native  RET 
representation  form  and  will  be  analyzed  using  RET  model 
facl I itles. 

Define  a  complete  set  of  functional,  performance,  Interface  and 
qualification  requirements  for  each  CSCI  including  programming 
constraints  and  standards,  design  constraints  and  standards, 
adaptation,  quality  factors  and  preparation  for  delivery. 


The  RET  process  model  Is  directly  responsive  to  most  of  the 
requirements  of  this  2167  activity.  Completeness  Is,  of 
course,  elusive.  Given  the  RET  model's  static  analysis, 
animation  and  prototyping  capabilities.  It  should  be 
possible  to  produce  a  set  of  requirements  which  are  as 
complete  as  current  understanding  of  the  system  and  Its 
environment  allows. 

Function  and  Interface  requirements  are  addressed  by  the 
model’s  language  for  representing  requirements  and  the 
methodology  for  Its  usage.  Performance  requirements  are 
addressed  by  the  model's  language  elements  which  permit  the 
representation  of  goals. 

Qualification  requirements  refer  to  tests  which  can  be  used 
to  validate  the  requirements.  Although  the  RET  model 
provides  validation  mechanisms  through  prototyping.  It  does 
not  explicitly  structure  test  sequences  (Inputs,  expected 
outputs,  coverage  criteria,  etc.)  which  are  designed  to 
demonstrate  requirements  compliance  during  final  acceptance 
testing. 

Since  the  RET  model  deals  with  candidate  solution 
architectures,  it  addresses  design  constraints  at  this 
prel imlnary  level.  It  does  not  address  programming 
constraints  or  standards  for  either  design  or  programming. 

Adaptation  to  changing  requirements  Is  implicit  In  the 
requirements  language  supported  by  the  model  and  explicit  In 
the  model’s  iterative,  refinement  nature. 

The  quality  of  products  produced  by  the  RET  process  model, 
although  not  explicitly  captured  In  the  model.  Is  a  research 
issue  to  be  addressed  during  the  long  term  construction  of 
the  RET. 

Identify  structured  requirements  analysis  tools  and  techniques. 

Implicit  In  the  RET  process  model. 

Conduct  Internal  In-process  reviews  of  requirements  documents  for 
the  purpose  of  Improving  the  qual ity  of  requirements  prior  to 
del Ivery  to  the  customer. 

The  RET  model  Incorporates  an  explicit  evaluation  step  as 
part  of  Its  Iterative  process  of  requirements  refinement. 
This  process  Includes  dynamic  analysts  of  requirements  and 
analysis  (performance,  cost,  risk)  and  prototyping  of 
partial  cesigns. 
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Software  Requirements  Analysis  Products  defined  by  2167 
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Updateo  versions  of  the  SDP,  SSPM,  SCMP  and  the  SQEP. 
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The  RET  process  model  does  not  address  these 
management  documents. 
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Updated  version  of  the  OCD. 
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The  domain  models,  which  are  major  elements  in  early 
RET  process  model  transformations,  could  provide 
valuable  mission,  application  and  user  cognitive 
Information  durtng  the  OCD  updating  process. 
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Reports  of  internal  reviews  for  recording  and  summary  purposes. 
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The  RET  process  model  does  not  address  management  reporting 
i  ssues. 

w*. 

A 

• 

System  Requirements  Specification  and  Interface  Requirements 
Specification  for  each  Computer  Software  Configuration  Item. 
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These  documents  could  benefit  directly  from  the  requirements 
and  partial  designs  which  are  the  major  products  of  the  RET 
process  model.  However,  several  format  and  content 
differences  must  first  be  reconciled  in  each  case: 
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The  SRS  requires  documentation  of  design  and 
implementation  Information  such  as  programming 
requirements  (language,  comp 1 1 er/assemb 1 er, 
programming  standards),  design  requirements  (sizing, 
timing,  standards,  constraints)  and  interface  requirem 
ents.  The  RET  model  clearly  does  not  address 
programming  Issues.  It  may  be  able  to  assist  with 
some  design  Issues  which  are  likely  to  surface  during 
preliminary  design.  Candidates  Include  performance 
constraints.  Interfaces  between  major  software 
components  (CSC Is)  and  Interfaces  between  major 
software  components  and  major  hardware  components. 
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The  actual  functional  and  performance  requirements 
certainly  could  be  directly  found  In  the  requirements 
and  partial  designs  produced  by  the  RET  process  model. 
However,  format  Is  a  potential  problem  because  the  SRS 
demands  that  these  descriptions  be  given  as  a 
functional  hierarchy  with  each  function  being 
described  by  Its  inputs,  processing  and  outputs.  The 
same  requirements  are  also  placed  on  all  subfunct tons. 
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Although  the  exact  representational  form  of  the  RET 
process  model's  requirements  has  yet  to  be  determined. 
It  is  likely  that  Is  underlying  model  will  be  object 
oriented.  In  this  case  some  translation  between  the 
objects,  relationships  and  operations  of  the  RET  model 
and  the  SRS  functional  breakdown  will  be  required. 

This  appears  to  be  more  than  a  minimal  effort,  since 
It  is  likely  to  require  human  interpretation. 


SRS  requires  low-level  operational  information  such  as 
site  dependent  environment  data  (e.g.,  radar  ranges), 
system  parameters  which  vary  according  to  operational 
needs  (e.g.,  allowable  trajectory  deviations)  and 
system  capacities  which  are  likely  to  change  (e.g., 
secondary  storage).  The  RET  model  is  not  Intended  to 
expose  Information  at  this  level  of  detail. 

The  SRS  includes  the  description  of  12  quality 
factors:  reliability,  correctness,  efficiency. 
Integrity,  usability,  maintainability,  testability, 
flexibility,  portability,  reusability  and  Inter¬ 
operability.  The  RET  process  model  deals  explicitly 
with  only  reliability.  Correctness  (consistency  and 
completeness  checking)  and  usability  (interface 
prototyping)  are  addressed  to  some  extent. 

The  SRS  includes  support  items  such  as  facilities, 
equipment,  software,  personnel  and  training  which 
would  be  used  during  the  development,  operation  and 
support  of  software  associated  with  the  requirements 
being  documented.  The  RET  process  model  does  not 
address  these  issues. 
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The  SRS  requires  qual if ication  of  the  software  which 
is  generated  from  the  stated  requirements  i.e.,  that 
it  actually  satisfies  the  requirements.  Qualification 
methods  Include  Inspection,  demonstration  (observable 
functional  operation),  testing  (collection  and 
subsequent  examination  of  data)  and  analysis 
(processing  of  accumulated  data).  The  RET  process 
model  supports  all  of  these  methods  to  some  degree. 

Inspection  and  demonstration  of  functional  operation 
can  be  done  through  rapid  prototyping  of  both 
Interface  and  functional  requirements.  The  RET  model 
provides  for  performance  and  reliability  analysis  of 
partial  designs.  Results  of  these  analyses  could  be 
helpful  In  supplementing  the  testing  and  analysis 
qualification  methods. 
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The  Interface  Requirements  Specification  describes  the 
requirements  for  one  or  more  Interfaces  between  major 
software  components  (CSCIs)  and  other  configuration 
Items  (software  and  hardware)  or  critical  Items  In  the 
system.  The  IRS  requirements  for  describing  the 
Interfaces  and  their  qualification  are  Identical  to 
those  contained  In  the  SRS.  The  IRS  Is  meant  to  be  an 
elaboration  of  the  SRS  with  respect  to  component 
Interfaces.  Consequently,  the  comments  given  above 
for  these  aspects  of  the  SRS  are  also  applicable  here. 

Software  Requirements  Analysis  Reviews  defined  by  2167 

2167  requires  a  Software  Requirements  Review  (SRR)  at  the 
completion  of  the  software  requirements  analysis  phase  to 
demonstrate  the  adequacy  of  the  OCD,  SRS  and  IRS. 

The  RET  model  defines  a  requirements  evaluation  activity  as 
part  of  Its  Iterative  process  for  refining  requirements  and 
partial  designs.  However,  the  Intent  of  the  RET  model  Is  to 
perform  evaluation  as  an  integral  part  of  the  requirements 
refinements  process.  This  differs  from  2167's  SRR  which  Is 
a  final,  formal  review.  Intended  to  convince  management  that 
the  requirements  are  acceptable  for  use  in  the  next 
(preliminary  design)  phase  of  the  software  development 
cycle.  In  the  sense  that  the  SRR  acknowledges  requirement 
problems  and  accommodates  their  resolution.  It  resembles  the 
RET  model's  evaluating  activity.  The  RET  model's  evaluation 
activity  could  be  used  as  a  single  review  point,  but  this 
would  have  to  be  considered  a  degenerate  case  of  RET  model 
app I icat Ion. 

Software  Requirements  Analysis  Baselines  defined  by  2167 

2167  defines  the  SRS  and  IRS  to  be  allocated  baseline  for 
further  design  and  Implementation  activities  at  the  time 
these  documents  are  accepted  by  management,  I.e.,  at  the 
successful  conclusion  of  the  SRR. 

The  RET  process  model  does  not  address  al location  or 
configuration  of  project  baselines. 


The  relationships  between  DOO-STD-2167  and  the  RET  process  model 
are  many  and  complex.  This  results  from  2167's  being  a  demanding 
standard  for  software  documentation.  Several  more  specific 
conclusions  are  possible: 


The  RET  process  model  clearly  does  not  satisfy  all  the 
documentation  requirements  of  2167.  As  would  be  expected. 
Its  major  deficiencies  are  related  to  2167's  management  and 
acquisition  requirements,  specifically  the  various  plans  for 
organizing,  procuring,  configuring,  baselining  and  assuring 
the  quality  of  a  large  scale  software  development  in 
accordance  with  established  standards  and  procedures. 

The  requirements  and  partial  designs  produced  by  the  RET 
process  model  can  provide  direct  assistance  In  preparing 
much  of  the  technical  Information  required  by  2167,  such  as 
the  functional,  performance.  Interface  and  qualification 
requ I rements. 

The  RET  process  model's  mission,  application  and  user 
cognitive  domain  models  can  be  used  to  generate  portions  of 
the  Operational  Concept  Document. 

The  RET  process  model  cannot  assist  in  the  analysis  of 
previously  developed  requirement  and  specification 
documents,  such  as  the  System/Segment  Specification,  as 
required  by  2167.  The  RET  model  does  not  recognize  the 
context  (earlier  software  development  cycle  phases)  In  which 
such  documents  were  developed,  nor  are  such  documents  In  the 
language  and  format  required  for  analysis  by  RET  processes. 

The  requirements  evaluation  activity  of  the  RET  process 
model  Is  iterative  in  nature  and  does  not  map  well  into  the 
single  review  point  mechanism  required  by  2167. 

The  RET  process  mode!  does  not  address  certain  lew  level 
design  and  Implementation  Information,  such  as  language  and 
compiler  specification,  which  2167  requires. 

The  RET  process  model  will  most  likely  be  based  on  a 
paradigm  (object  oriented)  which  will  not  be  isomorphic  to 
the  hierarchical  structuring  of  software  elements  as 
required  by  2167. 

The  RET  process  model  does  not  address  a  number  of  software 
quality  factors,  such  as  correctness.  Integrity  and 
maintainability,  which  2167  requires. 
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APPENDIX  C:  PROCESS  MODEL:  DETAILED  CHARACTERIZATION 


In  figure  3.1-1  of  section  3,  boxes,  clouds,  and  numbered  arcs 
represent  Information  relevant  to  the  engineering  of  requirements. 
In  this  subsection,  we  give  detailed  characterizations  of  each 
Information  type:  what  It  expresses,  its  role  in  requirements 
engineering.  Its  properties,  and  In  what  notations  it  is 
expressed.  We  consider  the  Information  types  In  the  order  we 
might  encounter  them  looking  down  figure  3.1—1 . 


Represented  as  a  cloud  because  of  its  external  and  meta-level 
status. 

Express.  There  are  external  constraints  and  concerns  Influencing 
requirements  engineering.  There  will  be  limitations  to  the  amount 
of  effort  that  can  be  spent  ("effort  constraints"  in  the  figure). 
Fully  engineering  a  "final  requirements  and  partial  solution 
architecture"  Is  only  an  Ideal,  so  It  is  Important  to  keep  a 
prioritized  list  of  objectives  In  hand  ("goals  of  requirements 
engineering  exercise"  in  the  figure). 

Role  in  requirements  engineering.  Focusing  In  on  the  RET  context, 
the  above  concerns  are  more  acute.  A  principal  goal  of  the  RET  is 
the  evaluation  of  requirements  engineering  tools  and  techniques 
(l.e.  measurement  of  process  and  product).  This  evaluation 
requires  many  RET  experiments,  each  of  limited  effort  and 
carefully  selected  objectives.  Hence  the  increased  acuteness. 
Concerns  of  making  a  limited  resource  (the  RET)  effective  to  a 
broad  audience  (Air  Force  requirements  engineers)  will  also 
Influence  the  nature  of  the  requirements  engineering  efforts 
("testbed  effectiveness"  in  the  figure). 


Represented  as  a  cloud  because  of  its  external  and  undocumented 
status. 

Express.  Wish  lists  are  expressions  of  desired  attainments,  but 
unlike  goals,  there  Is  generally  no  attempt  at  documenting  the 
context  of  the  desire.  Wishes  are  generally  characterized  as 
"likes"  and  "don't  likes"  in  the  domain-specific  terms  of  existing 
solutions.  Wishes  originate  with  mission  users  and  admin Istrators 
I n  the  field. 

Pole  In  Requirements  Engineering.  Wishes  motivate  the  need  for  a 
new  system;  they  are  an  external  source  in  the  process  model. 

They  are  not  formally  maintained  unless  they  can  be  precisely 
stated  In  goals. 
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Proper.t i es.  Very  Informal,  often  poorly  conceptualized.  Wishes 
may  be  Inconsistent,  even  If  from  the  same  source.  Worse  yet, 
they  may  be  misleading;  a  " I ike/dts I  Ike"  statement  Is  likely  to  be 
Imprecise  and  therefore  ambiguous.  Wishes  may  change. 

Notations.  Wishes  originate  In  people’s  heads.  Expressed  In 
natural  language.  Domain-specific  terminology  will  likely  be 
employed. 
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Express.  Goals  express  desired  attainments,  but  goal  expressions 
should  characterize  the  context  of  the  desire  too.  Target  system 
users,  administrators,  and  procurers  all  have  goals,  so  there  may 
be  many  sources  for  goals.  Normally,  each  set  of  goals 
characterizes  the  viewpoint  of  a  single  role  In  the  operation, 
administration,  or  maintenance  of  the  target  system.  Such  a 
characterization  Includes:  mission  performed.  Interactions  with 
target  system  and  other  users,  and  desired  attainments  (e.g. 
"immediate  response")  expressed  as  constraints  on  the  behavior  of 
the  target  system  (perhaps  thru  a  scenario). 

Goals  can  be  "guiding  philosophies",  documenting  the  metrics  of 
concern  (e.g.  "a  fast  tank").  In  addition  to  characterizing 
desired  system  behavior,  they  can  characterize  desired  cost  or 
desired  aval  lab  M Ity. 

Hole.  -La  Requirements  Engineering.  Goals:  (1)  Help  control 
complexity  early,  specifically  requirements  collection  and 
validation,  because  such  efforts  are  naturally  partitioned  Into 
different  viewpoints,  as  are  goals.  (2)  Goal  expressions  document 
the  source  and  context  of  expectations  on  target  system  behavior, 
which  sometimes  must  be  explored  during  reviews  or  during 
maintenance. 

Propert i es .  Within  a  set  of  goals,  consistency  should  be 
maintained,  but  between  sets  of  goals,  Inconsistencies  and 
conflicts  will  be  common,  reflecting  the  different  missions  In 
which  the  target  system  must  participate.  Concerns  of  technical 
achlevabl I ity  are  second-order. 

Notations.  Currently,  goals  are  rarely  expressed,  and  then  only 
In  descriptive  English  prose.  We  envision  use  of  doma in/m  I ss I  on- 
spec  If  Ic  formalisms  and  general  requirements  notations. 


Express.  Requirements  characterize  the  target  system  In  terms  of: 
its  operational  environment.  Its  Interfaces,  its  functionality, 
and  constraints  on  its  performance,  reliability,  development  and 
availability.  Requirements  express  “hat  ' s  needed,  but  also 
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document  assumptions  on  how  external  agents  (users,  existing 
systems)  will  Interact  with  the  target  system. 
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Role  In  requirements  engineering.  Obvious. 

Properties.  Requirements  should  be  operationally  testable  and 
consistent.  The  requirements  should  constitute  a  closed 
(complete)  external  model  of  the  target  system  and  Its 
environment.  Requirements  should  reflect  what  Is  technically 
achievable,  but  the  developer  has  full  Implementation  freedom 
within  the  limits  established  by  the  requirements. 

Not at  I ons .  Requirements  have  generally  been  Informal  and  stated 
In  a  natural  language,  but  future  requirements  specifications  will 
utilize  more  formal  notations  as  complexity  of  the  needs  to  be 
conveyed  Increases. 

Notational  mechanisms  are  often  employed  that  factor  requirements 
Into  different  concerns  and/or  levels  of  description. 


In  the  process  model,  a  scenario  Is  part  of  a  goals  or 
requirements  description. 

Express.  Scenarios  characterize  hypothetical  Interactions  that 
emphasize  target  system  role(s). 

Typically,  a  scenario  description  has  three  kinds  of  parts:  (1)  a 
characterization  of  the  initial  state  before  the  Interaction,  (2) 
a  sequence  of  actions  against  the  system,  and  (3)  a 
characterization  of  the  expected  results  (or  final  state). 
Scenarios  may  consists  of  several  action  sequence  and  expected 
result  pairs. 

Role  1 n.  r equ I  remen ts  eng  I neer 1 nq .  Scenarios  can  aid  the 
communication  of  goals  and  requirements  by  Illustrating  expected 
usage.  Or  scenarios  can  Illustrate  expected  system  behavior  In 
response  to  exceptional  or  stressful  Input,  thereby  having  the 
role  of  a  system  constraint.  Scenarios  can  be  used  as  test  data, 
especially  In  exercising  prototypes  and  In  system  acceptance 
tests. 

Propert I es.  Being  part  of  a  goals  or  requirements  description, 
scenarios  will  tend  to  exhibit  their  properties  In  roughly  similar 
degree,  e.g.  In  technical  ach levab 1 1 Ity  and  operational 
testability.  Scenarios  should  be  consistent  with  the  description 
of  which  they  are  a  part. 

Scenarios  may  be  fine-grained  In  action/result  characterization  or 
coarse-grained. 
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Notations.  Natural  language  and  domain-specific  terminology  are 
employed.  Some  formalization  of  their  structure  Is  not  uncommon; 
for  example,  representing  the  action  sequence  by  a  command  file 
(sequence)  of  procedure  calls. 


Express.  Solution  architectures  characterize  how  the  target 
system  can  satisfy  the  requirements.  Solution  architectures  are 
descriptions  of  the  target  system  as  a  composite  of  parts  (e.g. 
objects,  functions)  and  resources  (e.g.  people,  software, 
hardware)  and  how  the  different  parts  utilize  the  different 
resources. 

Role  In  requirements,  .eng  ineerlhg.  Aids  evaluation  of  requirements 
by:  (1)  aiding  understanding  feasibility  and  development 
Implications  (thru  cost  and  risk  analysis),  (2)  providing  a  basis 
for  prototyping  functionality,  Interfaces,  and  performance  which 
by  means  of  exercises  and  simulations  lead  to  a  better 
understanding  of  user  needs,  and  (3)  aiding  making  trade-off 
dec  Is  ions. 

Properties.  Solution  architectures  need  not  be  complete  "system 
parts  and  resources"  characterizations.  They  must  be  Internal ly 
consistent  and  should  be  consistent  with  the  goals  or  requirements 
they  were  constructed  to  stress. 

Notations.  Solution  architectures  are  mostly  formal  descriptions. 
Design  notations  may  be  utilized  for  expressing  the  partitioning 
of  the  system  Into  parts,  and  algorithms  for  expressing  resource 
utilization  strategies. 

As  with  requirements,  notational  mechanisms  are  often  employed 
that  structure  the  descriptions  into  partitions  and  levels. 


Generally,  domain  knowledge  Is  (cohesive)  knowledge  that  persists 
beyond  a  single  or  short-term  application.  Thus  If  collected, 
such  knowledge  can  be  reused  and  referenced  by  goals, 
requirements,  and  solution  architectures  of  several  projects. 

This  is  the  reason  for  its  separate  yet  special  status  (numbered 
arcs)  In  the  process  model  figure. 

Express.  A  domain  model  is  an  encoding  of  knowledge  specific  to  a 
domain.  The  knowledge  Is  characterized  thru  a  mixture  of 
terminology,  basic  facts,  and  empirically-derived  relationships. 
Comain  expressions  can  be  as  rich  and  complex  as  expressions  of 
knowledge.  In  general.  Knowledge  within  a  domain  may  exist  at 
different  levels,  e.g.  knowledge  of  how  to  use  other  domain 
knowledge  (I.e.  meta  knowledge). 


Role  In  requirements  engineering.  Domain  Information  has  value 
throughout  requirements  engineering. 

Examples  Identified  In  the  process  model  (not  a  complete  or 
mutually  exclusive  set):  (1)  Mission  models  -  the  goals  and  work 
style  of  a  particular  mission  user  (e.g.  Intelligence  analyst). 
Aids  In  creating  goals  descriptions.  (2)  Application  domain 
models  -  the  operations  available  In  a  particular  application 
(e.g.  text  editing).  Aids  In  understanding  user  I Ikes/dlsl ikes 
and  In  defining  systems  that  perform  similar  applications.  (3) 
User  cognitive  models  -  how  much  will  human  capabilities  (such  as 
decision  making)  be  stressed.  Aids  In  creating  and  analyzing 
requirements  descriptions.  (4)  Software  resources  (e.g.  operating 
system,  DBMS),  algorithms  (e.g.  search,  sort),  and  (5)  hardware 
models  (e.g.  networks,  run-time  environments,  and  performance 
attributes).  8oth  aid  In  designing  solution  architectures  and  in 
their  analysis. 

Properties.  As  noted,  domain  information  persists.  A  domain 
model  should  be  conceptually  cohesive  In  the  sense  of  constituent 
parts  relating  to  the  same  thing.  The  base  terminology  ("axioms") 
of  a  domain  may  be  selected  with  different  stability  and  fidelity 
In  mind. 

Notations.  Mostly  informal  and  natural  language.  As  knowledge 
representation  and  acquisition  technology  Improves,  use  of 
formalisms  In  expression  of  domain  Information  will  Increase. 


FINAL  REQUIREMENTS  AND  PARTIAL  SOLUTION  ARCHITECTURE 

The  final  set  of  requirements  and  architecture  ideally  are  the 
result  of  many  compromises  and  factor  In  many  concerns.  They  are 
the  end  product  of  the  requirements  engineering  process. 

Properties.  As  complete,  consistent,  feasible,  operationally 
testable,  and  traceable  to  user  needs  as  the  technology  and  time 
a  I  low. 


Process  Model  Activities  and  Support,  from  1930/1 335.. TflO-i .s. 

In  figure  3.1-1  of  section  3,  circles,  ovals,  and  rounded-corner 
boxes  represent  the  activities  relevant  to  the  engineering  of 
requirements.  In  this  subsection,  we  give  detailed 
characterizations  of  these  activities  In  terms  of: 
term  I  nation/ Invocation  criteria,  and  tools  that  will  assist  the 
activity  In  the  1990  and  1995  RET.  These  tools  are  products  of 
the  RET  R&D  program  and  are  described  in  greater  detail  In  section 
4  and  appendix  E.  The  1990  tool-support  characterization  Is  made 
in  terms  of  the  Evolutionary  track.  The  1995  characterization  Is 
made  In  terms  of  both  the  Evolutionary  and  Formal  Language  tracks. 
We  consider  the  activities  In  the  order  we  might  encounter  them 


looking  down  figure  3.1-1. 


Through  user  Interviews/reviews  and  reference  to  appropriate 
domain  models  (e.g.  mission  models),  wishes  are  refined  and 
documented  with  appropriate  context,  creating  goals. 


Termination  criteria:  Terminate  when  a  complete  and  consistent 
set  of  goals  has  been  constructed  for  each  viewpoint. 

Tool  support  In  1990  RET:  DBMS-based  mechanisms  to  support  reuse, 
syntax-directed  editing,  and  view  management.  The  Analyst  might 
be  enhanced  to  utilize  these  mechanisms. 

Tool .  support  in  19.95  RET:  Evolutionary  Track:  Advanced  DBMS  to 
support  access  and  manipulation  of  domain  models  by  both  users  and 
other  tools  and  to  help  state  goals.  Tools  that  support: 
knowledge  acquisition,  domain-specific  dialogues,  and  expert 
critlqu Ing. 

Formal  Language  Track:  Nothing  beyond  the  Evolutionary  Track 
approach  is  planned. 


Conflicting  descriptions  In  the  sets  of  goals  must  be  reconciled, 
creating  a  consistent  model  of  the  operational  environment  of  the 
target  system.  Expectations  of  target  system  behavior  are 
transformed  Into  precise  functional  requirements.  Scenario 
construction  and  analysis  may  aid  stating  the  nonfunctional 
requirements,  such  as  performance  and  reliability. 

Termination  criteria:  The  requirements  should  be  complete 
(relative  to  the  goals  and  scenarios  considered)  and  consistent. 

On  termination,  the  requirements  should  be  traceable  to  the  goals. 

Tool  support  In  1990  RET:  The  Analyst  tool  enhanced  with  DBMS- 
based  mechanisms  (see  "Goals  Synthesis  from  Wish  Lists"). 

Tool  support  in  1995  RET:  Evolutionary  Track:  (See  "Goals 
Synthesis  from  Wish  Lists".) 

Formal  Language  Track:  because  goals  and  requirements  are  stated 
In  the  same  formal  language,  no  language  change  Is  Involved  in 
going  from  goals  to  requirements.  Instead,  there  Is  selection  and 
refinement  of  goal  statements  via  a  methodology  for  formal 
requirements  synthesis. 


Static  analysis  can  reveal  some  types  of  Inconsistencies  and 
incompleteness.  If  goodness  meta-models  (e.g.  user  cognitive 


models,  models  of  safeness)  exist,  they  can  be  employed  to  derive 
properties  (e.g.  quality  of  cognitive  support,  degree  of 
safeness) . 

When  Invoked;  The  principle  to  be  followed  Is  that  any  kind  of 
analysis  that  can  be  done  automatically  should  be  done  as  early  as 
possible.  Whether  to  do  a  manual  analysis  requires  consideration 
of  the  Implied  value  added  and  effort  required. 

Tool  support  In  1990  RET:  The  Analyst  supports  some  consistency 
and  completeness  checking.  Almost  all  other  static  analysis  on 
goals  and  requirements  will  be  manual. 

Tool  support  in  .1995  RET:  Evolutionary  Track:  Consistency  and 
completeness  checkers  based  on:  (1)  a  controlled  natural  language 
representation  of  requirements,  and  (2)  domain  knowledge. 
Predictive  and  quality  metrics  and  tools  for  quality  assessment 
and  cr itlqu Ing. 

Formal  Language  Track:  The  use  of  a  formal  language  implies  that 
statements  are  formally  analyzable  and  that  proof  techniques  can 
be  used  on  the  underlying  predicates  to  determine  things  that  are 
subsumed  and  Incompatibilities. 


Dynamic  Analysis 

Dynamic  analysis  helps  one  to  understand  how  completely  and 
correctly  the  requirements  address  a  stated  concern  (e.g.  a 
reliable  Interface  to  a  set  of  sensors,  or  operator  training). 
Relevant  requirements  are  selected  and  a  walk-through  Is 
conducted.  Appropriate  tool  support  end  animation  would  be  very 
helpful. 

When  Invoked:  When  examining  requirements  for  goal  and  wish 
coverage  or  for  considering  spontaneous  or  new  concerns,  such 
analysis  Is  appropriate. 

Tool  support  in  1990  RET:  The  Analyst  tool  will  support  an 
automated  walk-through  of  requirements  (l.e.  an  analysis  of 
dataflows  through  the  various  user  viewpoints  which  Interact  with 
the  target  system)  and  animation  of  requirements  (i.e.  direct 
execution  of  user  or  system  actions  given  an  initial  state  and 
some  Inputs). 

Tool  support  in  1995  RET:  Evolutionary  Track:  An  animation-based 
dynamic  browsing  capability:  (1)  for  analyzing  requirements 
against  domain  knowledge,  and  (2)  for  analyzing  requirements 
interrel atlonsh ips. 


Formal  Language  Track:  Nothing  beyond  the  Evolutionary  Track 
approach  Is  planned. 


Candidate  Sol ut ton  Architecture  Synthes  La 

There  are  different  ways  to  partition  the  target  system  Into 
parts.  Different  resources  can  be  selected,  and  different 
resource  utilization  strategies  can  be  used  to  control  access  to 
the  resources.  Thus  the  principal  activity  Is  making  design 
decisions. 

Termination  criteria;  There  are  different  reasons  for 
synthesizing  solution  arch Itectures.  These  lead  to  different 
termination  criteria.  If  the  reason  ts:  (1)  Prototyping  -  we 
need  enough  architecture  to  produce  the  behavior  to  be 
Investigated.  (2)  Estimating  performance,  reliability,  or 
development  cost/risk  -  we  need  enough  architecture  to  make  the 
relevant  assessment.  (3)  Identification  and  definition  of  a 
satisfactory  solution  approach  -  we  need  to  define  an  architecture 
which  Is  sufficiently  complete  to  determine  with  high  confidence 
that  the  requirements  can  be  satisfied. 

Tool  support  in  1990  RET:  The  VHLL  Prototyping  tool  and  Rapid 
Prototyping  System  tools  will  provide  modest  design  capabilities 
such  as  support  for  functional  decomposition  and  allocation, 
concurrent  processing,  and  characterization  of  computer  systems 
and  communication  networks. 

Tool  support  in  1995  RET:  Evolutionary  Track:  Minimal,  although 
use  of  predictive  metrics  with  requirements  might  reduce  need  to 
do  solution  synthesis. 

Formal  Language  Track:  it  I s  poss I b I e  that  there  w 1 1  I  be  a 
capability  to  automatically  suggest  candidate  solutions,  but  this 
is  a  very  speculative  area,  end  is  not  a  focus. 
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After  defining  a  solution  architecture  that  produces  or  simulates 
selected  behaviors  of  Interest,  a  prototype  is  derived  and 
exercised  through  a  mixture  of  user-control  led  and  canned 
scenarios.  Thereby,  an  evaluation  of  the  selected  behaviors  is 
el ic Ited. 

Termination  criteria:  The  prototype  must  be  sufficiently  complete 
for  exercising.  The  exercising  must  be  sufficiently  thorough  to 
either  validate  the  selected  behaviors  or  provide  a  good  Idea  of 
what  are  the  alternative  candidates. 

Tool  support  in  19.90  RET:  For  1988:  Rapid  Prototyping  System 
tools  (RPS)  for  interpreting  prototypes  against  canned  scenarios; 
the  VHLL  System  Prototyping  tools  (VHLL)  for  executing  prototypes 
that  interact  with  users  or  simulations;  and  the  Analyst  tool  for 
scenario-based  symbolic  performance  estimates.  To  prototype  human 
interfaces:  especially  RPS,  but  VHLL  too.  To  prototype 
functionality:  VHLL.  To  prototype  for  performance:  all  tools. 
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For  analyzing  results  of  prototype  execution  against  a  scenario: 
each  tool  provides  a  specialized  analysis  capability.  A  database 
will  hold  results  and  data  management  facilities  will  aid  human 
analysis. 


Around  1990,  these  prototyping  capabilities  may  be  enhanced  with 
animation  and  with  dynamic  probes  to  determine  coverage.  There 
will  be  a  methodology,  possibly  metric-guided,  possibly  based  on 
CORE,  guiding  when  to  prototype. 


Shortly  thereafter,  a  knowledge-based  simulation  system  will  aid 
building  adaptive  scenarios  that  Interact  with  prototypes. 


Tool  support  In  1995  RET:  Evolutionary  Track:  Knowledge-based 
aids  for  evaluation  of  prototype  sensitivity  complemented  with: 
metrics-supported  analysis  and  a  methodology  for  when  to 
prototype.  A  static  coverage  tool  that  analyzes  the  prototype, 
determining  what  system  parts  are  exercised  by  the  scenario. 


Formal  Language  Track:  (1)  Th  i  solution  architecture  Is  the 
prototype.  When  driven  by  a  scenario,  the  prototype  generates 
behavior  which  is  automatically  checked  for  satisfaction  of 
requirements.  The  payoff  from  the  checked  prototype  Is  this: 
user-detected  undesired  prototype  behavior  Implies  Incorrect  or 
incomplete  requ i rements.  This  helps  the  user  carry  out  his  role 
of  checking  the  requirements.  (2)  Machine-assisted,  user-guided 
generation  of  scenarios. 
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A  candidate  solution  architecture  Is  subjected  to  analysis,  both 
manual  and  automatic,  to  determine  its  operational  goodness  (e.g. 
implied  performance  and  reliability)  and  development  goodness 
(e.g.  implied  cost  and  risk). 


When  Invoked:  When  a  candidate  architecture  Is  sufficiently 
complete  to  produce  a  high-confidence  analysis. 


Tool,  support  in  1990  RET:  The  Rapid  Prototyping  System  aids 
construction  of  usage-sensitive  performance  models. 


Tool  support  in  1995  RET:  Evolutionary  Track:  Metrics  and  tools 
for  estimating  cost,  time,  and  risk  from  solutions,  even  If 
solutions  have  a  distributed  or  knowledge-based  architecture. 
Tools  for  fault-handling  and  reliability  analysis. 


Formal  Language  Track:  The  use  of  a  formal  language  provides  a 
formal  basis  for  deriving  estimates  of  what  the  operational 
performance,  etc.,  will  be.  Otherwise,  does  not  offer  much  beyond 
the  Evolutionary  Track. 


Performing  dynamic  analysis,  rapid  prototyping,  and  analysis  of 
operat lon/development  Implications  leads  to  Insights  on 
requirements  correctness  and  alternatives.  These  Insights  suggest 
a  new/alternative  version  of  the  requirements.  Such  a  version  Is 
constructed. 

When  Invoked:  After  sufficient  analysis  of  the  current  version 
makes  clear  the  need  for  a  new/alternative  version. 

Tool  support  In  1990  RET;  Various  analysts  tools  and  a  CORE-based 
methodology. 

Tool  support  in  1995  RET:  Evolutionary  track:  Various  analysis 
tools  and  a  methodology,  possibly  metric-guided,  combining 
predictive  metrics  with  prototyping. 

Formal  Language  Track:  support  for  determining  whether  the 
requirements  are  satisfied  under  all  possible  conditions 
(scenarios  driving  the  solution  architecture).  Such  support  will 
be  based  on  a  statistical  and  symbolic  analysis  of  the  behavior 
space. 

In  the  RET  there  will  be  limitations  to  the  amount  of  effort  that 
can  be  expended  in  a  particular  requirements  engineering  exercise. 
There  will  be  a  list  of  specific  objectives  to  be  attempted  in  the 
exercise.  These  are  documented  in  the  Engineering  Context 
Descriptions,  which  controls  the  duration  of  the  exercise. 

Presumably  there  are  some  post-exercise  activities:  (1) 
Documenting  the  objectives,  constraints,  and  results.  (2) 
Evaluation  of  tools  and  techniques  used:  documenting  each  tool 
and  technique,  the  context  of  its  use,  problems  encountered,  and 
the  character  of  success. 

Tool  support  in  1990/1995  RET:  See  "Measurement  of  Process  and 
Prc juct"  below. 


Measurement  of  Process  and  Product 

This  activity  does  not  explicitly  appear  In  figure  3.1-1.  As  In 
the  case  of  the  "Engineering  Context"  cloud,  it  Is  a  meta-level 
activity.  As  a  key  objective  of  the  RET  is  the  evaluation  of 
tools  and  techniques,  this  activity  Is  included  here. 

RET  activities  must  be  carefully  planned,  metered,  and  measured  to 
assist  making  an  assessment  of  relative  tool/method  strengths. 
These  assessments  are  then  used  to  guide  development  of  new  tools 
and  techniques. 


*  For  each  agent/role: 

Role  description 

Eng  1  i  sh 

His  interaction  with  system 

His  Actions:  (Inputs/Outputs, 

Dataf low 

From/to  whom) 

Structured  English 

Control  constraints  on  actions 

Dataf low 

Activity  Attributes: 

Duration,  Frequency 

Operational  Cost 

Usage  statistics 

Structured  English  and  as 
Dataflow  annotations 

*  For  each  Datum  produced: 

Principal  structure 

Structured  English 

Datum  Attributes: 

S  i  ze 

Pers I stence 

As  annotations 

vt<l 


Dataf low 


*  Major  transaction 

Typical  usage 
Stressful  usage 
Transaction  Attributes:  As  Dataflow  annotations 

Duration,  Frequency 
Usage  Statistics 

Such  problem  domain  Information: 

*  Documents  the  environment  the  system  resides  In.  Documents 

system  activities. 

*  Collected  from  various  sources  (mission  users);  they  must 

val I  date  It. 

*  Aids  Identification  of  performance  and  reliability  stress 

points. 

The  1990  RET  will  aid  expressing  and  formalizing  this  information. 
There  will  be  checks  for  consistency  and  completeness.  Tools  will 
be  able  to  manipulate  such  information,  e.g.  for  performance 
prototyping:  estimating  target  system  performance  on  the  basis  of 
estimated  workloads. 
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Extrapolating  on  envisioned  1987/88  VHLL  Prototyping  tools  and  RPS 
functionality,  the  following  can  be  semi formally  expressed: 


Information: 

1.  System  Funct lonal Ity 
*  Funct iona I Ity  and  a 

solution  architecture 
that  real Izes  It 


*  VHLL:  Specification  by 
Nested  Dataflow  and 
Reusab I e  Modu I es 


2.  System  Interfaces 

*  Graphics,  dynamic  displays 


*  RPS:  Screen  contents  have 
Iconic  representation 
also  maps,  trajectories 


Interface  &  Functionality  Descriptions: 

*  Created  by  engineer  for  mission  user(s)  experimentation, 
comment,  and  validation. 


*  Exercised  partially  or  completely  against  scenarios  developed  as 
indicated  below  (under  ’’functional  model") 


Currently  RPS  supports  construction  of  four  types  of  performance 
models.  What  makes  these  RPS  models  different  from  VHLL 
prototypes  Is  that  Instead  of  associating  executable  code  with  a 
target  system  function,  a  resource  utilization  profile  Is 
associated  Instead.  Against  each  model,  scenarios  can  be 
exercised  and  performance  data  collected.  The  models  are: 

*  Model  of  Analyst  -  Represented  as  a  set  of  analyst  strategies. 

A  strategy  Is  represented  as  a  set  of  analyst-dependent 
procedures.  Each  procedure  Is  represented  as  a  sequence  of 
screen  Interactions.  Each  Interaction  Is  represented  as  a 
time  delay  and  a  target  system  task;  the  basis  for 
determining  the  resources  that  the  Interaction  consumes. 

Performance  can  be  measured  against  workstation  and 
processor  characteristics;  also  against  analyst 
characteristics. 

*  Model  of  Processor  -  The  processor  Is  represented  as  a  network 

of  computational  resources:  processors,  devices,  and  data 
I Ines.  Performance  character Istlcs  that  are  represented 
Include:  throughput,  capacity  rates,  transmission 
characteristics. 

The  network  is  exercised  against  workload  arrival  patterns. 

A  workload  is  represented  as  a  sequence  of  tasks  that 
contend  for  specified  processor  resources.  Performance  can 
thus  be  measured  against  processor/dev ice/operat I ng  system 
characteristics. 

*  Model  of  System  Function  -  The  target  system  Is  represented  as  a 

set  of  stimulus-thread  pairs;  In  which  the  stimulus 
determines  the  thread.  Each  thread  Is  represented  as  a 
schedule  of  logical  system  functions,  e.g.  database 
retrievals  and  Input/output  operations.  To  stimulate  the 
model,  a  scenario  can  be  defined.  A  scenario  Is  represented 
as  a  sequence  of  events.  An  event  Is  represented  as  a  set 
of  stimuli. 

The  system  Is  exercised  against  the  scenario.  Performance 
can  be  measured  against  target  system  functions 
character  1st Ics. 

*  Communications  Model  -  A  communications  system  Is  represented  as 

a  network  configuration  supporting  message  traffic. 
Performance  characteristics  that  are  represented  Include 
node  capacities  and  flow  rates. 

The  model  Is  exercised  against  a  message  stream.  Messages 
are  represented  as  a  source-destination  pair.  Performance 
can  thus  be  measured  against  communications  network 
characteristics. 


Though  wishes  will  change  with  time,  the  model  Indicates  there 
will  be  no  attempt  to  update  the  original  goals.  This  Is  because 
the  goals  become  folded  Into  the  Initial  requirements,  and  they 
get  updated  there  during  successive  cycles  of  requirements 
evaluation.  The  preferred  way  to  effect  permanent  updates  Is  thru 
changes  to  the  domain  models  referenced. 


The  model  Is  not  Intended  to  favor  a  particular  tool  or  technique 
within  generic  categories  of  support,  e.g.  prototyping.  It  Is 
therefore  largely  Independent  of  any  specific  tool  or  technique. 

Nor  Is  the  model  Intended  to  favor  a  particular  order  to  the 
requirements  engineering  activities.  Thus,  for  example,  an 
Interface  can  be  prototyped  before  performance  criteria  are  even 
established  In  the  requirements. 

With  a  very  few  exceptions  (namely,  "Testbed  effectiveness" 
information,  "Exit  Testbed"  and  "Measurement  of  Process  and 
Product"  activities),  the  process  model  Is  quite  generic  and  Is 
applicable  to  requirements  engineering  In  general. 


APPENDIX  D:  REQUIREMENTS  ENGINEERING  TESTBED  (RET)  TOOLS  AND 
ARCHITECTURE 


D.1  Tools  Currently  Under  Development 

These  are  the  tools  currently  under  development  by  RADC: 

"Analyst”  developed  by  Imperial  College  and  subcontractor 
Systems  Designers, 

"RPS"  (Rapid  Prototyping  System)  developed  by  Martin  Marietta 
Denver  Aerospace,  and 

"VHIL  Prototyping  Tools"  (VHLL  =  Very  High  Level  Language), 
developed  by  International  Software  Systems. 

Anal  yst 

Goal:  guide  the  requirements  analyst  in  the  col  lection, 
structuring,  and  validation  of  Information  about  target  system 
roles  and  context.  Result:  a  dataflow  specification  of  system 
requirements  organized  as  a  functional  hierarchy  of  viewpoints 
which  represent  the  processing  requirements  of  the  target  system, 
its  users,  and  other  systems  with  which  It  Interacts;  performance 
and  reliability  are  partly  addressed. 

Approach:  the  Analyst  supports  the  CORE  method  by  enforcing  CORE 
rules  that  guide  and  track  the  requirements  analyst  In  the  use  of 
CORE  diagrammatic  notations  in  the  statement  of  system  needs  and 
context. 

The  CORE  method  consists  of  rules  and  procedures  that  guide  the 
requirements  analyst  In:  factorizing  system  needs  and  context 
into  "viewpoints",  documenting  semlfornally  each  viewpoint,  and 
then  identifing  performance  and  reliability  needs  through 
consideration  of  transactions  or  scenarios  that  may  cross  several 
v I ewpoints. 

Each  CORE  viewpoint  is  documented  in  terms  of  Interactions 
(actions,  data  produced/consumed)  with  the  target  system  and  other 
relevant  agents.  A  viewpoint  is  also  produced  for  the  target 
system,  which  documents  the  functional  requirements;  as  opposed  to 
the  other  viewpoints,  which  model  expected  system  usage  and  the 
environment  of  the  target  system. 

Features: 

*  word  processing  support  is  a  key  component, 

*  extendible  rule  base,  heuristics, 

*  incremental  consistency  and  completeness  checking, 

*  Macintosh  host  and  Macintosh-style  graphical  interface, 

*  descriptions  In  terms  of  directed  graphs  with  textual 
labels  and  annotations. 


Future  research  to  be  pursued  (long-range  enhancements): 

*  Construction  (through  graphical  editing):  Much  leverage 

through  reuse.  Reuse  involves  finding  templates,  and 
Incorporating  (Instantiating  and/or  editing)  them. 

*  Analysis  (unsolicited  guidance):  Need  strategic  help  and 

active  guidance  (CORE  rules  provide  mainly  tactical 
guidance).  Need  better  sensitivity  to  development  history 
and  style. 

*  Navigation  &  Animation:  Support  arbitrary  browsing  (l.e. 

transitory  structuring  of  the  navigational  route). 

Support  walk-throughs  and  direct  execution  of  operational 
specifications.  Support  diagrammatic  interactions  with 
animation.  Support  rapid  prototyping. 

Database  implementation:  exploits  Prolog  mechanisms  for  tuple 
definition  and  pattern-matching  retrieval.  Performance  problems 
are  main  driver  for  future  enhancements. 


EES 

Goal:  Provide  the  Air  Force  mission  user  and  acquisition  engineer 
with  a  system  requirements  validation  capability  based  on  rapid 
configuration  of  user  interface  and  system  performance  prototypes. 
The  prototypes  are  constructed  through  a  minimum  development  of 
new  software  and  a  maximum  reuse  of  off-the-shelf  graphics  and 
simulation  modeling  packages.  The  RPS  provides  access  to  the 
modeling  approach  natural  and  appropriate  to  the  specific  user, 
and  orchestrates  the  execution  of  the  various  models  that  are 
produced. 

Approach:  The  following  models  can  be  produced: 

*  Mission  analyst  workstation  model:  helps  assess  performance 

Impacts  that  alternative  workstation  technologies  and 
procedures  have  on  the  analyst  mission. 

*  Processor  model:  helps  conduct  in-depth  resource  utilization 

studies  on  a  contemplated  architecture  under  varied 
work  loads.  A  basis  for  assessing  system  performance  under 
various  scenarios  and  Identifying  resource-critical 
conditions. 

*  Function  model:  helps  estimate  system  performance  given  an 

al location  of  system  functions  to  system  hardware  and 
software  resources. 

*  Communications  model:  helps  assess  performance  of 

contemplated  network  under  simulated  message  traffic. 

Features: 

*  RPS  interface  and  master  control  hosted  on  Apollo 


workstations, 

*  Color  and  dynamic  graphics  (animation). 

*  The  following  things  can  be  modeled  and  maintained  by  RPS: 

target  system  dataflow 
target  system  database 
target  system  knowledge  base 
display  Interfaces 

processor  and  communication  architecture 
scenarios  that  stress  the  entire  target  system,  Just 
the  communication  network,  or  Just  the  mission  of  one 
analyst 

Future  research  to  be  pursued  (long-range  enhancements): 

*  Addition  of  new  models. 


Database  Implementation:  The  RPS  databases  reside  on  different 
systems:  Apollo  workstations,  the  Vax  with  VMS,  and  a  Prolog/Ltsp 
processor.  No  attempt  will  be  made  to  Integrate  the  various 
databases.  The  RPS  user  can  access  these  databases: 


*  D3M  database  (Apol lo)  -  Used  to  model  target  system 

functionality  and  control.  Features:  menu-driven  schema 
definition,  report  formatter,  and  a  forms  generation 
package  supporting  the  interactive  creation  of  records. 

*  KEE  database  -  bu 1 1 ds/maintalns  knowledge  bases  In  support 

of  requirements  analysis  of  decision  support  systems. 

*  Switch  table  database  -  maintains  code  fragments  which 

associate  functionality  (e.g.  database  retrieval, 
simulation  model  execution,  dynamic  graphics)  with  a 
selected  region  (e.g.  graphic  depiction  of  a  system 
component,  portion  of  a  user  Interface  prototype)  of  the 
display. 

*  Support  tools  database  -  word  processor,  spreadsheet, 

report  generator,  and  version  manager  for  actual 
production  of  requirements  documentation. 


Goal:  Provide  a  capability  to  rapidly  and  minimally  describe  a 
program  that  executes  certain  target  system  functlon(s). 

Approach:  a  VHLL  (Very  High  Level  Language)  Is  used  for 
specifying  prototypes.  The  VHLL  prototyping  environment  provides 
support  for:  reuse  of  application-specific  modules,  rapid 
modification,  and  collection  of  performance  statistics  during 
execution.  Both  object-oriented  and  stimulus-response 
specification  approaches  can  be  used. 


Features: 

*  Hosted  on  Apol lo  workstation. 

*  A  browser  Interface  to  prototyping  and  project  data. 


instantiating  mechanisms  facilitate  reuse. 

*  Option  for  graphical  entry  of  VHLL  specifications. 

Future  research  to  be  pursued  (long-range  enhancements): 

*  VHLL:  enhance  with  a  richer  standard  set  of  data  and 

function  operators. 

*  Prototyping  environment:  embed  within  a  general  design 

database  In  which  programs  would  be  directly  synthesized. 
Tools,  display  would  be  initiated  by  database  activity. 

Database  approach: 

*  Database:  will  maintain  function  interface/body 

descriptions,  data  types,  prior  designs,  classifications, 
queries. 

*  Data  model:  an  object-centered  data  model  supporting  CAD 

p  ids  for  dynamic  metadata  and  data  with  complex  attribute 
values.  Query  language:  selection  allows  navigation  over 
relationships.  Support  for  updates  thru  views. 

*  Design  management:  maintain  consistency  of  designs  and 

tracks  dependencies.  Versioning:  will  be  fine-grain; 
there  may  be  multiple  active  versions.  Configuration 
management  to  manage  consistent  assemblies  of  objects. 

*  Strategy:  Utilize  commercial  CAD  database.  Database  is 

system  focus  for  all  synthesis,  analysis  activities. 


A  very  loose  coupling  between  the  contractors’  tools  Is  feasible 
within  the  time  and  scope  of  the  current  tool  development  efforts. 

Loose  coupling  means  that  the  data  produced  by  one  tool  can  be 
input  to  another  with  only  minimal  (perhaps  no)  effort  and  minimal 
(perhaps  no)  loss  of  Information.  Loose  coupling  will  allow  the 
testbed  user  to  take  (say)  a  functional  dataflow  representation 
created  by  one  tool  and  provide  it  as  input  to  another  tool. 

Loose  Coup  I Ing  game  plan: 

(1)  Each  contractor  will  provide  an  ob ject  analysis  on  his 
tools.  Identifying  objects,  attributes,  relationships, 
operations,  and  aggregation  operations  (building  bigger 
objects) . 

(2)  Identify  semantic  gaps  In  the  different  object  analyses.  (A 
semantic  gap  between  two  tools  indicates  precisely  what  kind 
of  information  they  cannot  share  and  which  must  be  lost  In 
moving  shared  data  between  them  because  of  a  lack  by  one  of 
the  tools  of  the  necessary  semantics  for  representing  the 
information.)  Where  few  gaps  exist,  produce  an  object 
schema  overlapping  all  three  tools. 


(3)  The  common  schema  will  be  the  basis  for  determining  the  kind 
of  file  conversions  that  must  be  provided.  In  order  to  move 
Information  successfully  from  one  tool  to  the  other. 

(4)  Other  support: 

*  for  file  transfers  between  Macintosh  (hosting  the 

Analyst)  and  the  Apollo  (hosting  RPS,  VHLL  tools). 

*  some  configuration/version  management  assistance  across 

the  tools  is  desired. 


Integration  Strategy 

To  significantly  reduce  the  amount  of  effort  required  to  achieve 
integration,  the  panel  defined  a  strategy  of  things  to  be  done  in 
the  very  near  term  by  RADC  and  Its  tool  contractors.  The  strategy 
is  to:  (1)  Define  standards  for  the  final  Integrated  RET  of  1990. 
(2)  Make  current  tool  contractors  aware  of  these  standards  to 
influence  the  design  and  Implementation  of  their  tools.  (3) 
Encourage  and  maintain  informal  communication  between  the  tool 
contractors  and  RADC  to  help  track  problems  associated  with  the 
use  of  these  standards. 

To  achieve  (1)  and  (2)  above,  all  current  tooi  contractors  were 
Invited  to  a  meeting  of  the  panel  to  help  define  the  standards. 
Conclusions  are  presented  below.  To  help  achieve  (3),  contractor 
meetings  to  discuss  database  Issues  have  been  recommended  to  begin 
in  early  1987. 

Following  the  discussion  on  standards,  the  loose  coupling  plan  (of 
section  D.2)  Is  contrasted  with  the  Integration  strategy. 
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Assumption:  If  all  current  tool  contractors  can  agree  to  a 

similar  Interface  style  now  (I.e.  how  to  Invoke  tools,  how  to 
manipulate  data),  the  later  Integration  will  be  much  smoother,  and 
In  the  Interim,  users  going  from  one  tool  to  another  will  not  have 
to  fundamentally  change  the  style  In  which  they  access 
requirements  and  tool  functionality. 

Current  tool  Interfaces:  Graphical  editing  plays  a  key  role  In 
every  contractor's  approach  to  the  tool  Interface. 

Standards  to  be  fol lowed: 

(1)  Macintosh  (Apple"*"^)  Style  Manual  documents  syntactic 

conventions  for  menu  systems  and  supports  the  paradigm  of 
direct  manipulation  at  the  Interface.  Conventions  outlined 
In  this  manual  are  to  be  used  by  tool  contractors  to  achieve 
a  common  syntax  to  their  menu  systems. 


Standards  desired: 


TM 

(1)  Paradigm  of  resource  files  (Windows  )s  conventions  for 
exploiting  menus,  dialogue  boxes.  Icons,  and  accelerators 
for  the  purpose  of  creating  programs  with  adaptable 
interfaces  (l.e.  without  need  of  recompiling). 

(2)  On  commands:  consistency  of  application-specific  windows 
(e.g.  the  command  bar)  and  the  commands  (and  their  options) 
invokable  within  them  (e.g.  same  file  operations  and 
operation  names). 

(3)  On  •’gestures":  consistency  In  the  Implicit  ways  of  getting 
things  done  (e.g.  expand/contract  boxes). 

(4)  Standardization  vehicle:  Systems  Designers  can  forward 
appropriate  Analyst  code/conventions  to  the  other  tool 
contractors  as  a  guide. 

The  panel's  user  Interface  Integration  strategy  Is  to  have  a 
common  ed Itor/browser  interface  to  the  objects,  through  which 
different  actions  can  be  selectively  Invoked.  This  Is  consistent 
with  the  panel's  recommended  RET  architecture. 

Database  Standards,  and  Strategy 

Assumption:  The  different  contractors  are  developing  tools  that 
need  to  access  (share)  the  same  data. 

Thus  standards  are  needed  for: 

( 1 )  Data  Model . 

*  How  Is  the  data  structured?  What  part  is  formal  structure 
(e.g.  arcs),  and  what  part  is  text  (e.g.  nodes). 

*  How  Is  the  data  accessed?  Need  standards  on  the  query  and 
programmatic  Interfaces. 

*  What  atomic  data  and  database  operations  on  these  data  are 
assumed?  Example:  establish/delete  node/arc. 

*  Associative  retrieval:  need  standards  on  how  to  navigate 
associations  and  whether  It  Is  permitted  In  both  directions. 

(2)  Schema. 

Should  the  schema  Itself  be  represented  In  the  database  so 
that  tools  can  operate  on  It  (e.g.  to  track  inheritance)? 
What  are  the  performance  implications  of  this  strategy  (i.e. 
each  data  access  requires  a  schema  access)? 


(3)  Rules  and  Inferencing. 

Types  of  rules:  (1)  design  constraint  rules  to  maintain  the 
consistency  of  the  objects  being  built,  (2)  transformation 
rules  which  permit  derivation  of  Implied  Information  from 
multiple  relationships  In  the  data,  and  (3)  rules  as  a  form 
of  program  encoding  (capturing  programs  In  an  Incremental 
and  modi f lable  way). 

*  Rule  representation:  How  should  they  be  declared?  Should 
they  be  embodied  In  the  code  or  represented  In  the  database 
to  be  manipulated  as  data? 

*  Support  which:  der Ive-as-needed  or  truth  maintenance? 

(Given  rule  A,B  — >  C,  and  A,B,  do  you  assert  C  and  store  It 
In  the  database  or  derive  It  as  needed?) 

*  How  to  turn  rules  on/off? 

(4)  Triggering. 

When  a  database  update  matches  a  particular  pattern,  do  some 
processing.  Example:  update  screen  as  result  of  a  change. 

*  Similar  Issues  as  in  "Inferencing"  above. 

*  Two  different  notions  of  "transaction": 

**  atomic  transaction:  "batching"  all  updates  to  avoid 
triggering  rules  until  completion,  and 

**  Interactive  transaction:  updates  are  Immediate,  but  if 
an  abort  occurs  before  completion  of  the  transaction, 
the  database  Is  restored  to  the  pre-transaction  state. 

(5)  Import/export  facility. 

An  Import  facility  allows  Inputlng  data  produced  by  another 
tool  In  some  foreign  format.  An  export  facility  allows 
taking  out  data  in  a  format  (e.g.  ASCII)  that  can  be  used  by 
other  tools. 


Problems  and  future  research  Issues: 

*  Limited  database  size:  Artificial  Intelligence  databases 
are  capable  of  handling  small  numbers  of  facts  (about  30K). 
How  does  their  performance  scale  up? 

*  Performance: 

**  Strategies:  Cache.  Optimize  for  retrieval 
operations.  Limit  size  of  the  problems  to  be 


addressed  by  the  tools  (l.e.  In  the  context  of  RET 
experiments) . 


**  Performance  problem:  The  problem  of  determining  the 
context  In  which  to  bring  something  In  from  disk  is 
exacerbated  by  the  large  number  of  rules.  Inheritance 
mechanisms  and  having  the  schema  represented  in  the 
database  further  exacerbates  this  problem. 


Availability  to  testbed: 


Loose  Coup  I  I ng:  Early  1988,  with  nominal  effort. 
Integration:  1990,  Involving  considerable  effort. 


Degree  of  common  data  representation: 


Loose  Coupling:  Perhaps  none,  file  conversion  probable. 
Integration:  Common  representation  for  all  data  shared. 
Storage/retr feval  thru  a  common  database. 


Shifting  between  tools: 


Loose  Coupling:  Limited  In  frequency,  as  conversion 
results  in  some  loss  of  sem antic  content  and  affects 
performance. 

Integration:  Frequent.  The  RET  user  will  be  able  to  take 
full  advantage  of  equal  and  uniform  access  to  all  tools 
and  objects,  yielding,  for  example,  improved  prototypes 
and  increased  productivity. 


We  present  below  a  long-range  (at  least  ten  years)  software 
architecture  for  the  RET.  The  architecture  is  made  up  of  three 
parts:  a  user  Interface  (the  "RET  editor"),  a  database  (the  "RET 
database"),  and  a  language  (the  "RET  language"). 


The  RET  architecture  consists  of: 


RET  editor  -  the  user  Interface  to  al I  documents  and  tools.  The 
interface  can  be:  customized  (e.g.  domain-specific,  dialog  style, 
graphics  vs.  text),  syntax-directed,  and  tolerant  to  ambiguity. 
Capab i I ities  for; 


(1)  Browsing  (finding)  relevant  objects. 


(2)  Viewing  (selection/presentation)  objects. 


(3)  Annotating  objects  with  comments,  questions,  etc. 


(4)  Invoking  all  RET  tools. 


iifiS 


IHMWS 


(5)  Multiple  w tndows/contexts/user  access. 

(6)  Direct  editing  of  relevant  objects,  WYSIWYG  style. 

These  capabilities  require  close  coupling  between  the  RET  editor 
and  RET  database. 

RET  Database  -  the  common  repository  for  all  requ I rements-rel evant 
Information,  all  RE  tool  artifacts.  Including  evolution. 

Features: 

(1)  Object  definition  facility,  based  on  the  RET  language. 

(2)  Query/update  interface,  supporting  associative  retrieval. 

(3)  Rule-based  architecture.  Provision  for  defining  rules  to 

obtain  consistency,  effect-propagation,  and  automation. 

Is  fundamental  to  long-range  RET  goal:  a  WYSIWYG  sup- 
pressed-tool  interface,  in  which  tools  that  process 
objects  are  directly  invoked  by  state  changes,  as 
indicated  by  the  rules. 

(4)  View  management.  Support  for  view  customization. 

(5)  Performance  scales  with  size  well;  simultaneous  access. 

(6)  Multi-language:  available  to  all  RET  tools. 

PET  Language  -  a  single  formal  w i de-spectrum  language  for 
expressing  requirements,  specifications,  scenarios,  and  domain 
definitions.  Goals  and  partial  descriptions  are  given  formal 
semantics.  Support  required: 

(1)  customization  of  the  user  interface  to  make  the  language 

usable  to  a  broad  user  class  (RET  editor), 

(2)  support  and  tracking  of  Incremental  modifications  to 

descriptions  (RET  database  and  RET  editor). 
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APPENDIX  E:  DETAILED  DESCRIPTIONS  OF  RET  R&D  ISSUES 


This  appendix  contains  detailed  characterizations  of  all  R&D 
Issues  that  form  part  of  the  RET  R&D  program. 

RET  R&D  program  research  Issues  and  development  issues  are  given 
respectively  "RET  Research  Issue"  and  "RET  Development  Issue" 
characterizations,  and  have  the  following  format: 

which  track  (formal  language  vs.  evolutionary), 
problem  description, 

value  in  requirements  engineering  process, 
solution  approach, 

5-year  objectives, 

10-year  objectives, 
risk  (likelihood  of  results), 
cost  (degree  of  effort  required), 
contingencies  (dependencies), 
conclusion  (recommendations),  and 

author  name  (which  panel  members  wrote  the  description). 

Those  issues  considered  to  be  the  best  candidates  for  funding  were 
also  given  "RET  R&D  effort"  characterizations.  An  RET  R&D  effort 
characterization  has  this  format: 

objective  (what  is  to  be  accomplished), 
scope  (effort  boundary,  technical  areas  tc  be  addressed 
and  those  which  will  not), 

technical  approach  (how  the  objective  will  be  achieved), 
background  (relevant  terminology,  related  work  toward 
this  effort’s  objective,  etc.), 
references, 
duration  (In  months), 
cost  (in  doll ars) 

deliverables  (product  and  date  (i.e.  months  from  effort 
start),  and 

roadmap  (task  schedule  against  1986-1995  timeline). 

Costs  Include  capital  costs  for  equipment  and  resources.  These 
estimated  costs  reflect  the  panel  perception  that  these  R&D 
efforts  require  the  highest  quality  people  and  laboratories. 

Section  E.1  characterizes  the  R&D  Issues  addressed  in  the 
evolutionary  track.  Section  E.2  characterizes  the  R&D  issues 
addressed  in  the  formal  language  track.  As  all  formal  language 
track  issues  contribute  to  the  same  goal,  namely  the  formal 
treatment  of  requ i reri.ents,  a  "research  issue"  characterization  and 
roadmap  is  given  for  the  whole  track;  individual  Issues  are  only 
given  R&D  effort  character izat Ions. 


Evolutionary  Track 


PROBLEM  DESCRIPTION: 

The  synthesis  activity  takes  unstructured  textual  expressions  of 
need  as  Inputs  (often  conflicting  with  each  other)  from  different 
sources  and  produces  as  output  a  formal  structuring  of  the  input 
texts  (which  have  been  refined,  made  feasible,  and  conflicts 
resolved).  Synthesis  Is  a  complex  activity,  often  a  group 
activity,  and  for  the  foreseeable  future,  the  challenge  Is  one  of 
supporting  manual  synthesis. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Explicit  requ I rements  statements  have  a  critical  communication 
role  between  and  among  system  definers,  designers,  and  reviewers. 
They  form  the  end  product  of  the  RET  activities. 

SOLUTION  APPROACH: 

(1)  Reuse  mechanisms:  The  RET  analyst  will  need  to  access, 
modify,  and  Incorporate  existing  requirements  descriptions.  This 
requires  mechanisms  for  browsing  previously-synthesized 
descriptions,  and  composing  them  Into  the  new  synthesis  context. 

(2)  Viewing  mechanisms:  As  synthesis  is  largely  manual,  the  RET 
analyst  will  need  to  see  all  relevant  information  (mission 
viewpoints,  domain  Information,  and  related  descriptions).  Some 
(possibly  most)  Information  will  be  developed  using  domain- 
specific  terminology  and  notations.  Controlling  information 
changes  and  presentations  thru  different  perspectives  of  a  group 
Is  largely  a  problem  of  view  management. 

(3)  Syntax-directed  editing  mechanisms:  Syntax  aids  and  checking 
should  be  well  Integrated  with  editing,  supporting  the  synthesis 
of  the  formal  parts  of  goals/requirements  descriptions. 

5  YEAR  OBJECTIVES: 

Provide  an  RET  database  of  appropriate  data  definition, 
query/update  Interface,  and  view  management  facilities. 


Enhance  the  Analyst:  Research,  develop,  and  Integrate  better 
syntax-checking,  view  management,  and  reuse  mechanisms;  base  o 
top  of  the  RET  database. 

10  YEAR  OBJECTIVES: 

Improved  classification  of  requirements,  domain  Information, 
etc..  Improving  their  ease  of  accessibility. 

Develop  multi-language  synthesis  capability:  enabling  a 
heterogeneous  group  of  people  to  develop  portions  of  the  same  set 
of  requirements  utilizing  different  domain  languages. 


RISK: 

To  the  extent  the  objectives  can  be  achieved  thru  database 
technology,  there  Is  a  high  probability  of  success.  Otherwise, 
there  Is  significant  risk. 

COST: 


1-2  man  years  for  5-year  objectives.  2-3  man  years  for  the  10- 
year  objectives. 

CONTINGENCIES: 

Finding  an  appropriate  RET  database  In  less  than  5  years. 
CONCLUSION: 

To  the  extent  that  the  viewing  mechanism  can  be  provided  by  a 
DBMS,  it  should  be  brought  into  the  RET  with  no  specific  research 
funding.  Where  new  research  is  needed.  It  Is  not  clear  that  the 
technical  opportunities  are  well  understood,  so  funding  cannot  be 
recommended  at  this  time. 

AUTHOR  NAME: 


Michael  Konrad,  Terry  Welch. 
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Evolutionary  Track. 


PROBLEM  DESCRIPTION: 


Domain  information  can  be:  (1)  persistent  -  useful  to  more  than 
one  project  and/or  for  long  periods  of  time,  and  (2)  a  basis  for 
decisions  -  key  assumptions.  But  domain  knowledge  is  often  In 
people’s  heads  (e.g.  mission  specialists).  Clearly,  collecting 
and  organizing  domain  Information  and  then  making  that  Information 
accessible  to  both  RET  engineers  and  tools  presents  enormous 
challenges  and  opportunities. 

VALUE  IN  REQUIREMENTS  PROCESS: 

The  most  significant  role  played  by  domain  Information  is  as 
assumptions  or  facts  about  the  environment  In  which  the  target 
system  is  to  reside  (e.g.  user  characteristics  and  expectations). 

SOLUTION  APPROACH: 

The  technical  approach  must  focus  on  the  use  of  domain  knowledge 
In  requirements  engineering;  much  effort  Is  already  being  spent  on 
the  general  problem.  The  effort  must  determine:  (1)  what  domain 
knowledge  Is  used,  (2)  how  It  is  used  (for  what  purpose  (including 
by  tools)  and  with  what  frequency),  and  (3)  how  It  changes  (nature 
and  frequency  of  change).  In  other  words,  build  a  usage  model  of 
domain  knowledge  In  requirements  engineering.  Only  then  can  a 
practical  structure  and  representation  for  domain  information  be 
selected. 

Ways  of  getting  new  knowledge  Into  the  model  (knowledge 
acquisition  and  classification,  domain-specific 
I  anon. nr e/ ' nterfaces)  and  getting  It  out  (Interfaces  with  tools, 
do'  expert  crltiqu Ing,  and  domain-specific  I anguage/ Interfaces) 
w  e  tailored  to  these  results  but  need  investigation.  It  Is 
not  yet  c  ear  which  Input/output  approach  will  provide  the  best 
leverage.  An  input/output  approach  will  be  recommended  at  the 
conclusion  of  the  study  described  In  the  preceding  paragraph. 

5  YEAR  OBJECTIVES: 

Selection  of  domain  model  structure: 

*  structure  driven  by  key  requirements  engineering  concerns 

*  goal s/requ i rements/sol ut Ion  architecture  descriptions  and 


►  vs 


their  rational Izatlons  can  reference 

*  basis  for  recommending  Input/output  approach  to  getting 

domain  Information  In/out 

Build  models  of  one  or  two  key  domains 
Adapt  tool  Interfaces  to  permit  access 
10  YEAR  OBJECTIVES: 

Enhance  RET  tools  to  better  Interpret  and  use  domain  knowledge. 

Build  a  domain  Information  Input/output  mechanlsm(s)  out  of  some 
combination  of: 

*  classification  scheme  for  automatic  acquisition, 

*  domain-specific  I anguage/ Interface, 

*  domain  experts  that  can  assist  the  engineer  In: 

eliciting  requirements  relevant  to  that  domain  and 
In  the  use  of  domain  knowledge 

Augment/revise  domain  model  structure  to  capture  key  domain 
Information  required  by  new  tools. 

RISK: 

Low  to  moderate  risk.  Understanding  the  general  role  of  domain 
knowledge  in  requirements  engineering,  both  present  and  future,  is 
key  to  success.  Failure  to  understand  that  role  may  lead  to 
capture  of  Irrelevant  Information.  New  tools  may  out  date  the 
usefulness  of  the  knowledge  represented.  Domain  knowledge  is  also 
subject  to  change  with  Improved  theories.  Automatic  acquisition 
Is  largely  a  classification  problem,  and  poses  some  significant 
risk. 

COST: 

Degree  of  effort  required:  5-year  objectives:  4  man  years  for 
first  two  objectives,  3  man  years  for  last  one.  10-year 
objectives:  10  man  years. 

CONTINGENCIES: 

Tied  to  evolution  of  tools  and  domain  theories. 

CONCLUSION: 

The  domain  model  approach  offers  much  promise,  even  If  automatic 
domain  knowledge  acquisition  should  prove  very  hard  and  therefore 
remain  significantly  manual.  Recommend  funding. 

AUTHOR  NAME: 

Michael  Konrad,  Terry  Welch. 


ISSUE  NAME:  Domain  Model s/ 1 n format  Ion 


OBJECTIVE: 

Define  a  domain  modeling  technology: 

(1)  helps  organize  domain  Information  Into  a  model, 

(2)  for  access  by  tools,  and  references  by  Goals  and  ... 
Requirements  descriptions, 

(3)  for  commun Icatlng  domain  information  to/from  humans, 

(4)  supports  domain  model  evolution. 

Interface  tools  to  domain  models.  Enhance  tool  use  of  domain 
information. 


SCOPE: 


Technical  areas  to  be  addressed:  Domain  models.  Knowledge 
acquisition.  However  the  technical  approach  is  not  to  advance 
generic  technologies  In  these  areas  but  to  develop  new 
technologies  addressing  the  most  pressing  problems  in  the  use  of 
domain  Information  In  requirements  engineering. 

TECHNICAL  APPROACH; 

Determine  the  usage  of  domain  information  in  requirements 
engineering  as  a  basis  to  identifying  where  the  best  leverage  lies 
(what  to  capture  and  how  to  structure  If  for  later  access).  Build 
domain-specific  languages  and/or  interfaces  for  the  most  common 
domain(s)  to  aid  in  capturing  the  domain  Information  relevant  to 
an  application.  Address  relevant  database  issues:  synergies 
between  different  domains,  aggregation  and  classification, 
versioning,  and  evolution.  Enhance  appropriate  tool  Interfaces. 
Investigate  role  for  expert  systems  that  use  domain  Information  to 
help  elicit  and  validate  requirements. 

BACKGROUND: 

The  "Analyst"  [lj,  a  requirements  engineering  tool  under 
development  by  Systems  Designers  (an  effort  sponsored  by  RADC), 
already  provides  a  limited  formal  modeling  capability.  Imperial 
College,  under  contract  to  RADC,  Is  exploring  enhancements  that 
would  increase  the  conceptual  modeling  capability  of  the  tool  C23» 

Several  reports  discuss  the  capture,  use,  and/or  evolution  of 
domain  knowledge  in  software-related  activities:  requirements 
engineering  C3],  automatic  programming  [4].  Several  reports 
discuss  the  development  of  tools  to  aid  encoding  domain  knowledge 
In  the  creation  of  "friendly"  domain-specific  interfaces  and 


languages:  for  domain-specific  requirements  languages  [5],  and 
for  the  construction  of  software  systems  [63  and  [7j.  Reports 
that  stress  the  role  and  value  of  domain  modeling  as  a  precursor 
to  the  "requ irements  specification"  include  [33  and  [8], 

REFERENCES: 

[1]  M.  Stephens  and  K.  Whitehead,  "The  'Analyst'  -  An  Expert 
Systems  Approach  to  Requirements  Analysis",  draft. 

[23  A.  Flnkelstein  and  C.  Potts,  "Structured  Common  Sense: 

The  Elicitation  and  Formalization  of  System 
Requirements",  draft. 

[33  S.  J.  Greenspan,  J.  Mylopoulos,  and  A.  Borgida,  "Capturing 
More  World  Knowledge  In  the  Requirements  Specification", 

Proc.  6th  Int.  Conf.  Software  Eng.  Also  other  papers  by 
the  authors. 

[43  David  R.  Barstow,  "Domain-Specific  Automatic  Programming", 
IEEE  TSE,  November  1985. 

[53  Alan  M.  Davis,  "The  Design  of  a  Family  of  Application- 
Oriented  Requirements  Languages",  IEEE  Computer,  May 
1982. 

[63  James  M.  Neighbors,  various  Draco  papers. 

[73  S.  Sundfor,  two  reports  on  application  of  Draco:  "Draco 
Domain  Analysis  for  a  Real  Time  Application  ..."  Irvine 
TPs:  RTP  015-016. 

[83  Bartlett,  Cherrie,  Lehman,  MacLean,  and  Potts,  "The  Role 
of  Executable  Metric  Models  In  the  Programming  Process", 
Imperial  College  of  Science  &  Technology. 

DURATION: 

Domain  model Ing  Study  30  months 

Select  domain  model  &  build  example(s) 

Interfacing  tools  to  domain  model  18  months 

Enhanced  tools  (part  of  separate  tool  efforts) 

18  months 

Build  Input/output  mechanism  for  domain  Information 

30  months 

Evolve  domain  model  structure  24  months 

COST: 

Domain  modeling  Study  .6  million 

Interfacing  tools  to  domain  models  .45  million 

(attached  to  "Tools  Integration  contract") 

Enhanced  tools  .45  million 

(attached  to  "Tools  Integration  contract") 


Build  Input/output  mechanism: 
Evolve  domain  model  structure 


75  mill  Ion 
.3  mill  Ion 


DELIVERABLES: 

PROOUCT  DATE  (months  after  start): 


Domain  model Ing  Study 


Domain  Model  Structure  Design 

18 

months 

Build  example  domain  models 

27 

months 

Domain  Model  Structure  Ftnal  Destgn 
(Attached  to  integration  contract) 

30 

months 

Design/del Iver  Interface  to  domain  models 

18 

months 

Bui 

Enhance  to  better  exploit  domains 

Id  Input/output  mechanism 

36 

months 

Design 

18 

months 

Evo 

Del  I vered  code 

Ive  domain  model  structure 

30 

months 

Revised  Domain  Model  Report 

16 

months 

Design  and  delivered  code  modifications 

24 

months 
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Evolutionary  Track 


PROBLEM  DESCRIPTION: 

Analyze  the  requirements  without  executing  a  prototype  of  the 
solution.  Provide  checks  for  completeness  and  consistency. 

Animate  the  requ irements. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Requirements  analysis  at  an  early  phase  avoids  difficulties 
associating  requirements  with  solutions  and  reduces  the  need  for 
scenarios.  Static  analysis  removes  inconsistencies  at  an  early 
stage  in  requirements  engineering. 

SOLUTION  APPROACH: 

Convert  the  requirements  to  a  controlled  natural  language.  (A 
near  English-like  language  with  a  restricted  vocabulary).  Check 
the  expressions  in  this  language  for  consistency  and  completeness. 
Through  a  domain-specific  meta-knowledge  base  question  untestable, 
contradicting  or  redundant  requirements.  Animate  the  requirements 
by  allowing  editing  based  on  graphical  representat Ions  of  the 
controlled  natural  language  expression  organized  along  differing 
categories. 

5  YEAR  OBJECTIVES: 

Develop  a  controlled  natural  language  and  accompanying  meta¬ 
knowledge  base. 

10  YEAR  OBJECTIVES: 

Continue  5  year  development  with  the  addition  of  user  Interfaces 
and  analysis  tools. 

RISK: 

Excellent  chance  of  results. 


COST: 


6  man  years  for  the  5  year  objective. 

3  man  years  for  the  10  year  objective. 


CONTINGENCIES: 


Dependent  on  controlled  natural 
CONCLUSION: 

Fund  research  at  a  low  level. 
AUTHOR  NAME: 

Stephen  Sherman 


I  anguage. 


Not  top  priority 


ISSUE  NAME:  Requirements  -  Static  Analysis 
OBJECTIVE: 

Provide  tools  to  analyze  requirements  without  executing  a 
prototype  of  the  solution. 

SCOPE: 

The  effort  is  bounded  by  the  need  to  formalize  the  requirements 
language.  As  more  semantic  and  syntactic  restrictions  are  placed 
on  the  requirements,  we  approach  the  concepts  of  the  formal  track 
and  may  borrow  from  the  formal  track  effort. 

TECHNICAL  APPROACH: 

Provide  the  minimum  formal Ization  necessary  to  carry  out  the 
solution  approach.  If  the  funding  for  the  formal  track  Is  not 
timely,  this  effort  may  provide  a  start  on  the  formal  tack. 

BACKGROUND: 

The  relevant  technology  Is  the  theory  of  languages  and  logic 
(consistency  and  completeness). 

REFERENCES: 

For  animation:  CORE/Analyst  development  system  by  Imperial 
College  and  Systems  Designers.  For  Controlled  Natural  Language: 
"Requirements  for  Mechanical  Translation  -  Problems,  Solutions, 
Prospects"  in  Feasibility  Study  on  Fully  Automatic  High  Quality 
Translation,  (Stachowitz,  Bar-Hillel,  Winograd,  Bob  Simmons) 
Austin,  Texas,  Linguistics  Research  Center,  The  University  of 
Texas  at  Austin,  RADC-TR-7 1-295,  1971. 

DURATION: 

Control  led  Natural  Language  (CNL)  -  3  years 
User  Interfaces  and  Analysis  ( U I A ) —  2  years 

COST: 

Controlled  Natural  Language  (CNL)  .9  million 
User  Interfaces  &  Analysis  (UIA)  .45  million 

DELIVERABLE: 


CNL 

Design 

6  months 

CNL 

Prototype 

18  months 

CNL 

System 

36  months 

U!  A 
Ul  A 


Des Ign 
System 


6  months 
2  years 


REQUIREMENTS  -  STATIC  ANALYSIS  R&D  EFFORT 


Evolutionary  Track. 


PROBLEM  DESCRIPTION: 


As  a  result  of  both  RET  experience  and  outside  advances,  new 
requirements  analysis  capabilities  will  be  brought  Into  the  RET. 
The  RET  requirements  engineer  will  need  guidance,  e.g. 
methodologies.  In  the  Intelligent  application  of  these  new  RET 
capabilities  to  solve  his  requirements  problems. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Intelligent  guidance  In  the  application  of  new  and  better  tools 
and  techniques  will  lead  to  automation  of  more  requirements 
analysis  resulting  In  improved  requ i rements  and  productivity. 

SOLUTION  APPROACH; 

The  problem  of  giving  guidance  to  requirements  analysis  activities 
will  be  attacked  two  ways. 

in  the  near  term,  the  CORE  method  will  be  extended  to  include 
guidance  in  the  use  of  currently-planned  tools  (e.g.  the  "RPS"  and 
"VHLL  Prototyping  tools"). 

In  the  long  term,  the  role  of  CORE  in  guiding  requirements 
analysis  activities  needs  to  be  reexamined;  other  methodologies 
should  be  considered.  The  selected  methodology  must  gu!c'n  all 
engineering  activities  that  help  effect  analyses  on  requirements, 
e.g.  solution  architecture  synthesis  to  derive  cost,  risk,  and 
performance  estimates. 

5  YEAR  OBJECTIVES: 

Extend  the  CORE  method: 

*  into  solution  architecture  synthesis  and  analysts,  and 

*  to  provide  guidance  in  use  of  RPS  and  VHLL  Prototyping 

tool s. 


10  YEAR  OBJECTIVES: 

Provide  an  RET  requirements  analysis  methodology: 

*  structures  analysis  activities  to  meet  RET  user  objectives. 


E-15 


*  guides  solution  architecture  synthesis,  and 

*  supported  by  RET  formalisms  and  tools. 


RISK: 


COST: 


Moderate  risk.  Selection  of  the  best  methodology  will  be 
difficult  because  of  the  difficulty  In  determining  the  relative 
effectiveness  of  tools  and  techniques. 


Degree  of  effort  required:  5-year  objectives:  2  man-years.  10- 
year  objectives:  6  man-years. 


CONTINGENCIES: 


Knowing  the  good  vs.  bad  tools  and  techniques  helps  one  define  a 
good  methodology.  Thus  strongly  dependent  on  success  in  the 
"Testbed  effectiveness"  R&D  effort.  Testbed  tools  effectiveness 
measures  must  be  developed  and  RET  usage  data  collected  both  on 
the  tools  and  on  the  applications  and  purposes  of  use. 


CONCLUSION: 


Good  guidance  in  the  use  of  RET  tools  is  a  key  to  successful 
requirements  analysis.  We  recommend  funding  this  effort. 


AUTHOR  NAME: 


Michael  Konrad,  Terry  Welch. 
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ISSUE  NAME:  Requlrenents  Analysis  Methodology 

OBJECTIVE: 

Provide  RET  users  guidance  on  the  use  of  new  RET  tools,  through: 

(1)  Near  term:  extending  the  CORE  method  Into  strategies  for 
using  RPS  and  VHLL  Prototyping  tools. 

(2)  Long  term:  select  a  methodology  that  directs  all 
engineering  activities  related  to  requirements  analysis. 

SCOPE: 

Technical  areas  to  be  addressed:  Requirements  Analysis, 
Requirements  Methodology. 

TECHNICAL  APPROACH: 

Near  term:  Do  (1)  below.  Then  select  those  CORE  steps  that  can 
be  made  more  effective  through  utilizing  RPS  and  AND  Prototyping 
tools  capabilities.  Place  hooks  In  the  Analyst  to  Indicate  when 
to  use  these  prototyping  tools  and  pass  data  to  them. 

Long  terms:  Do  (1),  (2)  and  (3). 

(1)  Analyze  each  RET  tool,  identifying  what  it  analyzes  and  what 
is  produces.  Assess  the  utility  of  each  tool  by  analyzing  RET 
tool  effectiveness  data  If  available. 

(2)  Identify  a  few  candidate  RET  methodologies  (In  particular, 
CORE);  extend  them  to  appropr iatel y  utilize  the  tools  determined 
in  Cl),  Identifying  tool  entrance  and  termination  conditions. 
Determine  experimentally  their  relative  strengths  and  weaknesses. 
Select  one  methodology  to  be  the  "RET  methodology". 

(3)  Build  a  tool  that  Indicates  to  the  RET  user  which  RET  tools 
are  appropriate  at  each  stage  (if  more  than  one,  what  value  and 
effort  are  implied),  consistent  with  the  RET  methodology. 

BACKGROUND: 

The  "Analyst"  £1]  Is  a  tool  supporting  CORE  requirements 
specification  and  analysis  activities.  It  currently  does  not 
address  prototyping. 

SREM  [2]  was  an  early  requ I rements  methodology.  In  a  recent 
evaluation  C3!3,  SREM  was  considered  to  be  best  used  when  defining, 
correcting,  or  analyzing  software  specific  requirements;  i.e. 
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after  the  (traditional)  system  requirements  analysis  phase  had 
been  completed,  and  long  after  the  conceptual  phase  (e.g. 
feasibility  assessment  and  trade-off  analysis).  The  methodology 
proposed  by  this  research  would  address  all  of  these. 

REFERENCE: 

Cl]  M.  Stephens  and  K.  Whitehead,  "The  Analyst  -  A  Workstation 
for  Analysis  and  Design",  Proc.  8th  Int.  Conf.  Software 
Eng.,  IEEE  Comp.  Soc.  Press,  1985. 

C2]  M.  Alford,  "A  Requirements  Engineering  Methodology  for  Real 
Time  Processing  Requirements",  IEEE  T.S.E.  SE— 3 Cl). 

C3]  A.  Stone,  D.  Hartschuh,  and  B.  Castor,  "SREM  Evaluation", 
Rome  Air  Development  Center,  RADC-TR-83-314. 

DURATION: 

Extended  Core:  add  prototyping  12  months 

RET  methodology:  all  analysis  activities  36  months 


COST: 


Extended  CORE:  add  prototyping  .3  million 

RET  methodology  .9  million 

DELIVERABLES: 

PRODUCT:  DATE  (months  after  start): 

Extended  CORE: 

Report  on  adding  prototyping 
Del i vered  code: 

Hooks  in  Analyst  to  prototyping  tools 

RET  methodology: 

Report  on  RET  tools: 

Scope  and  Effectiveness  12  months 

Report  on  methodology  comparison  24  months 

Del Ivered  code: 

Indicates  which  RET  tools  to  use  36  months 


12  months 
12  months 
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PROBLEM  DESCRIPTION: 


The  dynamic  analysis  of  requirements  constitutes  the  most  valuable 
means  for  validating  a  candidate  set  of  requirements.  It  provides 
end  users  and  system  specifiers  with  feedback  and  allows  them  to 
"exercise”  certain  characteristics  of  a  proposed  new  system.  This 
early  feedback  will  greatly  enhance  the  quality  and  fidelity  of 
future  systems. 

VALUE  IN  REQUIREMENTS  PROCESS: 

As  referenced  above,  early  feedback  on  the  validity  and  accuracy 
of  system  requirements  Is  highly  desirable.  It  will  enable 
substantially  more  of  the  logical  "What  If-ing"  that  leads  to 
improved  systems  designs  and  implementations.  Early  detection  of 
faulty  assumptions  and  assurance  that  the  requirements  are  clearly 
and  concisely  represented  will  result  in  considerable  cost  and 
time  savings  for  major  programs. 

SOLUTION  APPROACH: 

This  dynamic  analysis  of  requirements  Is  clearly  very  open.  A 
number  of  capabilities  are  suggested.  A  user  would  clearly  like 
to  be  able  to  Investigate  the  cohesiveness  and  validity  of  a 
candidate  set  of  system  requirements.  The  use  and  continued 
development  of  rapid  prototyping  systems  to  study  a  system  is 
clearly  a  part  of  what  Is  required.  Augmentation  of  initial 
prototyping  capabilities  with  the  ability  to  "exercise"  a  proposed 
system  and  study  the  Internet  at ionsh ips  of  its  various 
requirements  is  highly  desirable.  Packaging  this  type  of  animated 
prototyping  capability  In  a  manner  In  which  end  users  and  their 
analysts  can  study  a  proposed  system  Is  the  direction  of  most 
promising  pursuit. 

Augmenting  these  prototyping  capabilities  with  domain-specific 
knowledge  based  concepts  is  a  more  long-range  undertaking. 
Developing  techniques  for  representing  and  analyzing  new  system 
requirements  versus  a  known  domain  model  and  browsing  through 
various  Interrelationships  could  be  very  valuable.  A  phased 
approach  to  this  Is  suggested  since  there  are  clearly  quite  a  few 
aspects  to  this  overall  problem.  The  first  phase  of  this  activity 
will  be  aimed  at  determining  the  best  ways  to  capture  and 
represent  the  user's  problem  domain  with  suggesting  approaches  to 
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its  analysis.  Following  a  go/no-go  decision,  much  more  effort 
will  be  focused  on  building  a  more  powerful  browsing  and  analysis 
capability.  This  advanced  support  system  would  allow  the  building 
of  much  more  complete  user  domain  models  and  provide  more  powerful 
•’logical  browsing"  and  analysis  capabilities  for  more  completely 
validating  end  user  requirements. 

A  central  goal  In  all  of  these  activities  is  to  provide  a  much 
more  flexible  environment  for  specifying  and  analyzing  higher 
level  requirements  and  specifications.  By  providing  more  rapid 
and  valid  feedback  at  an  early  stage  In  a  project  vastly  superior 
systems  can  be  developed  with  substantially  less  risk. 

5  YEAR  OBJECTIVES: 

Develop  general  purpose  prototyping  capabilities  that  will  allow  a 
user  at  a  very  high  level  to  graphically  specify  a  series  of  tasks 
and/or  activities  and  some  of  their  important  interrelationships 
and  "execute"  thru  animation  the  flow  of  information  and  control 
between  tasks. 

10  YEAR  OBJECTIVES: 

Develop  knowledge-based  systems  which  allow  end  users  and  their 
support  specialists  to  specify  attributes  of  their  problem  domain 
and  query  a  system  which  "exercises"  queries  about  related 
attributes  and  specifications  for  new  systems.  For  example, 
dynamic  browsing  Is  of  the  form,  "I  am  a  pilot  charged  with 
carrying  out  a  certain  type  of  mission,  what  requirements  might  be 
affected  by  this  particular  type  of  mission?" 

RISK: 

Rapid  prototyping  systems  are  becoming  available  already,  thus  the 
risk  associated  with  the  5  year  activities  seems  quite  minimal. 

The  longer  range  activities  are  much  more  questionable  since  they 
really  depend  rather  heavily  upon  significant  advances  in 
knowledge  collection  and  manipulation. 

COST: 

Rapid  prototyping  and  animation  activities  should  be  sponsored 
with  a  5  -  10  person  year  effort  in  the  first  5  year  time  frame. 

An  intelligent  knowledge-based  capability  should  be  investigated 
in  phases  to  determine  its  feasibility  and  expected  payoff.  An 
initial  effort  of  2  person  years  should  be  supported  to  be 
fcl Icwed  up  with  a  go/no-go  decision  for  a  subsequent  5-10  person 
year  effort  in  the  10  year  time  frame.  (If  something  reasonable 
can  not  come  from  this  level  of  effort  -  we  are  probably  trying  to 
picneer  the  field  too  much.) 


The  prototyping  activities  are  currently  tied  to  various 
representations  and  me-' hodo I og  I es .  Since  it  is  not  clear  there 
exists  any  UNIQUE  desirable  methodology  we  should  attempt  to 
develop  tools  that  are  adaptable  to  methodological  changes.  It  Is 
also  clear  that  graphical  methods  will  continue  to  evolve  and  we 
must  be  supportive  of  such  changes  as  well. 

As  we  look  into  more  sophisticated  knowledge-based  approaches  to 
representing  and  analyzing  high  level  specifications,  we  become 
very  dependent  upon  advances  in  the  representation,  collection, 
and  manipulation  of  these  knowledge  bases.  We  should  focus  on  the 
use  of  this  technology  rather  than  sponsoring  its  research 
directly. 

CONCLUSION: 

Support  for  enhanced  rapid  prototyping  is  clearly  doable  and 
valuable.  Exploration  of  techniques  for  providing  more  user 
''animation"  types  of  capabilities  in  conjunction  with  rapid 
prototyping  is  also  highly  desirable. 

A  phased  approach  should  be  applied  to  the  longer  range  knowledge- 
based  type  activities  since  their  feasibility  and  payoff  is  not 
yet  clear. 
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ISSUE  NAME:  Dynamic  Analysis 
OBJECTIVE: 

The  objective  of  this  research  area  Is  to  develop  Improved 
methodologies  and  tools  for  analyzing  and  validating  system 
requirements  at  a  very  early  life  cycle  phase.  Clearly  the  more 
feedback.  Iteration,  and  analysis  that  can  be  performed  within  the 
requirements  activity  Itself,  the  more  cost  and  schedule  savings 
can  be  real Ized. 

SCOPE:  Two  tvpes  of  activities  are  Included  In  this  area. 

The  first  Involves  the  development  of  enhanced  prototyping 
capabilities.  Most  of  the  ideas  involved  here  have  In  fact  been 
demonstrated  to  varying  degrees  already.  No  real  surprises  are 
anticipated  In  their  further  development  and  application  on  this 
effort. 

The  second  focus  area  for  research  Is  less  well  defined.  Although 
the  desire  to  apply  knowledge-based  technology  to  this  problem 
appears  useful,  the  practicalities  of  this  technology  have  yet  to. 
be  total  I y  proven. 

TECHNICAL  APPROACH: 

The  proposed  technical  approach  will  be  evolutionary  In  nature. 
Initial  emphasis  will  be  placed  on  Incorporating  "known" 
techniques  and  types  of  tools  within  a  framework  suitable  for 
requirements  analysis  activities.  This  framework  will  be 
augmented  with  increasingly  powerful  analysis  capabilities. 
Hopefully,  with  time,  "smart  aids"  will  serve  as  guides  to  the 
user  In  better  representing  and  analyzing  future  system 
requ I rements. 

The  application  of  dynamic  analysis  to  requirements  engineering, 
as  a  general  area  of  Investigation,  Is  clearly  very  open  ended.  A 
number  of  capabilities  are  suggested.  A  user  would  clearly  like 
to  be  able  to  Investigate  the  cohesiveness  (consistency  & 
completeness  -  validity)  of  a  candidate  set  of  system 
requ I rements. 

Task  1 : 

The  use  and  continued  development  of  rapid  prototyping  systems  to 
study  a  system  is  clearly  the  first  task  to  be  Included. 
Augmentation  of  initial  prototyping  capabilities  with  the  ability 
to  "exercise"  a  proposed  system  and  study  the  Interrelationships 
of  Its  various  requirements  Is  highly  desirable.  An  Integrated 
prototyping  system  will  be  designed  and  built  Incorporating  this 
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animation  capability  and  packaging  it  for  use  by  end  users  and 
their  analysts. 

Task  2  [Phase  Ij]: 

Augmenting  these  prototyping  capabilities  with  domain-specific 
knowledge-based  concepts  Is  a  more  long-range  undertaking. 
Developing  techniques  for  representing  and  analyzing  new  system 
requirements  for  a  known  domain  model  and  browsing  through  various 
Interrelationships  can  be  very  valuable.  A  phased  approach  to 
this  is  suggested  since  there  are  clearly  quite  a  few  aspects  to 
this  overall  problem. 

Task  2  [Phase  I  1 3 

Thus,  the  second  major  phase  of  this  activity  will  be  aimed  at 
determining  the  best  ways  to  capture  and  represent  the  user's 
problem  domain  with  suggested  approaches  for  its  analysis.  A 
moderate  feasibility  study  and  initial  demonstration  task  will  be 
followed  up  with  a  go/no-go  decision.  Assuming  a  favorable 
decision,  a  subsequent  effort  will  focus  on  building  a  more 
powerful  browsing  and  analysis  capability.  This  advanced  support 
system  will  support  the  building  of  much  more  complete  user  domain 
models.  It  will  also  provide  more  powerful  analysis  and  "logical 
browsing"  capabilities  for  validating  end  user  requirements. 

Central  to  all  of  these  activities,  is  a  strong  desire  to  provide 
a  much  more  flexible  environment  for  specifying  and  analyzing 
higher  level  requirements  and  specifications.  By  providing  more 
rapid  and  valid  feedback  at  an  early  stage  in  a  project,  vastly 
superior  systems  can  be  developed  with  substantially  less  risk. 


BACKGROUND: 


Historically,  the  broad  effects  and  high  costs  of  detecting  and 
correcting  requirements  errors  or  inconsistencies  late  in  the  life 
cycle  have  been  clearly  documented.  The  earl ier  their  detection 
the  better  -  thus,  there  Is  a  very  natural  desire  to  detect  such 
errors  as  early  as  possible  In  the  life  cycle.  Early  error 
detection  and  classification  activities  focused  on  analyzing 
actual  program  code.  Many  ideas  have  been  developed  which  if 
applied  even  earlier  In  the  lifecvcle  could  produce  substantial 
savings  of  time  and  effort. 
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[in  particular  his  rapid  prototyping  system  with  its  animated 
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CContalns  Interesting  Ideas  for  representing  and  analyzing  dynamic 
properties  of  systems.  The  specific  examples  are  aimed  at  the 
coding  level  -  but  the  approaches  to  dynamic  analysis  and 
potential  extension  to  earlier  phases  of  the  life  cycle  are 
clearly  suggested. U 


DURATION: 


Task  1 : 


24-36  months 


Task  2  [phase  1 3 :  18  months 

[phase  I  1 3:  36  months 


Tasks  1 : 


Task  2  [phase  1^]: 


[phase  II [3 : 


DELIVERABLE: 


PRODUCT: 


Task  1 : 


Improved  prototyping  systems 
($750,000  to  $1,500,00) 

Feasibility  Study  &  Simple  Prototype 
($300,000) 

Initial  Operational  Analyst  Assist 
($750,000  to  $1,500,000) 


DATE: 

(months  after  start) 


Requirements  for  Prototyping  System 
Preliminary  Design/Test  Plan 
Detailed  Design/Test  Procedures 
Program  Code/Users  Manuals 
Test  Results  Summary 
Concept  Paper  for  Future  Research 


Task  2  [Phase  lj: 

Requirements  for  Knowledge-Based  System 

Preliminary  Design/Test  Plan 

Detailed  Design/Test  Procedures 

Program  Code/Users  Manuals 

Test  Results  Summary 

Concept  Paper  for  Phase  I  I  Research 


4  mos. 
8  mos. 
14  mos. 
20  mos. 
26  mos. 
36  mos. 


4  mos. 
8  mos. 
12  mos. 
16  mos. 
22  mos. 
24  mos. 


Task  2  [Phase  !  l[: 
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Requirements  for  Knowledge-Based  System 
Preliminary  Design/Test  Plan 
Detailed  Design/Test  Procedures 
Program  Code/Users  Manuals 
Test  Results  Summary 
Concept  Paper  for  Future  Research 


4  mos 
8  mos 
14  mos 
20  mos 


26  mos 
36  mos 
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RET  Research  Issue.  Evolutionary  Track. 


PROBLEM  DESCRIPTION: 

The  synthesis  activity  takes  system  requirements  as  Input  and 
produces  as  output  a  design  (solution  architecture);  both 
represented  as  structured  text.  This  Involves  normal  aspects  of 
hardware/software  sel ectlon/development. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Synthesis  of  designs:  (1)  aids  performance  analysis,  (2)  aids 
examination  of  functionality  alternatives  (mission-oriented 
individuals  need  to  "try  It  out"),  and  (3)  gives  Inslqht  into 
feasibl I ity. 

SOLUTION  APPROACH;  There  could  be  three  kinds  of  assistance: 

(1)  Expert  Advisor:  providing  tactical  guidance  on  resource 
selection  and  management. 

(2)  Design  Crftfquer:  evaluates  the  architecture  goodness  and 
design  style  (cohesion,  complexity). 

(3)  The  second  approach  is  to  provide  a  specialized  design 
language  (VHLL)  in  which  designs  would  be  specified. 

Issue  (1)  Implies  use  of  resource  models  and  both  (1)  and  (2) 
require  a  fairly  formal  representation  of  requirements. 

5  YEAR  OBJECTIVES: 

Enhanced  RPS  and  VHLL  Prototyping  tools:  to  aid  synthesis: 
better  v lew/reuse/syntax  mechanisms. 

10  YEAR  OBJECTIVES: 

Expert  advisor. 

Design  critlquer. 

Specialized  design  language  (VHLL). 

RISK: 

10-year  objectives:  Medium/high  because  of  lack  of  insight. 
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.  Evolutionary  Track 


PROBLEM  DESCRIPTION: 

Create  a  scenario  that  drives  the  prototype.  The  scenario  may 
Indicate  specific  actions  expected  of  the  prototype  for  checking 
results.  The  scenario  should  exercise  the  prototype,  yet 
Intelligently  react  to  the  prototype's  responses  (e.g.  play  the 
role  of  an  antagonist).  Scenario  creation  should  also  be 
sensitive  to  the  type  of  data  required  by  the  prototype. 

A  coverage  analysis  tool  Is  needed  to  Indicate  the  parts  of  the 
prototype  that  were  brought  Into  play  during  Its  elocution,  the 
requirements  that  are  related  to  each  part,  and  se,  ,'tivlty  of  the 
prototype  to  scenario  Input. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Scenarios  play  a  key  role  as  drivers  of  prototypes  in  experiments; 
often  providing  the  only  way  to  evaluate  a  system  operationally. 
Tools  for  scenario  creation,  coverage,  and  analysis  enhance  the 
experimental  usefulness  of  prototypes. 

SOLUTION  AFPROACH: 

Scenario  generation  and  scenario  coverage  require  different 
technological  solutions.  Below  we  examine  the  Issues  In  this 
order:  (1)  generation  of  adaptive  scenarios  and  driving  the 
prototype,  (2)  probing  the  prototype  for  coverage,  (3)  given  the 
expected  system  responses  under  a  scenario,  attempt  to  prove  that 
the  requirements  and  solution  architecture  satisfy  the  scenario 
and  derive  the  coverage,  and  (4)  how  these  approaches  compare  to 
what  the  Rapid  Prototyping  System  tools  (RPS)  offer. 

(1)  GENERATION  OF  ADAPTIVE  SCENARIOS  AND  HOW  TO  DRIVE  THE 
PROTOTYPE 

Adaptive  scenarios  are  viewed  as  consisting  of  two  kinds  of  very 
high-level  descriptions:  (1)  strategy:  what  the  scenario  Is 
trying  to  accomplish  against  the  (target)  system  and  (2) 
resources:  what  resources  the  strategy  has  available  to  carry  out 
Its  objectives.  The  corresponding  view  of  prototypes  is  that 
there  Is  a  prototype  environment  consisting  of  (1)  the  prototype 
code  and  (2)  a  high-level  representation  of  resources  (especially 
sensors  and  actuators)  and  two  maps:  (2a)  how  system  resource 
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changes  (e.g.  sensors')  map  to  specific  prototype  Inputs 
(translation  to  concrete  data  and  associating  the  data  with 
prototype  code'  parameters/ports)  and  (2b)  another  map  of 
prototype  outputs  to  system  resource  changes  (e.g.  actuators'). 

Thus  before  an  adaptive  scenario  can  drive  the  prototype,  the 
following  must  be  specified:  (1)  scenario  strategy  and  resources, 
(2)  system  resources  and  how  they  map  to  prototype  Inputs/outputs, 
and  (3)  how  scenario  resource  actions  affect  system  resources,  and 
vice  versa.  Then  to  drive  the  prototype,  all  resources  are 
simulated,  and  the  following  sequence  is  repeated: 

*  the  scenario  strategy  determines  what  changes  to  make  in  the 

activities  of  Its  resources, 

*  these  activity  changes  drive  changes  In  system  sensors, 

*  sensor  changes  are  mapped  into  prototype  inputs, 

*  the  prototype  executes,  creating  output, 

*  the  prototype's  outputs  are  mapped  into  actuator  changes, 

*  actuator  changes  drive  changes  In  scenario  resources. 


On  each  successive  iteration,  the  strategy  examines  how  Its 
resources  have  been  affected  by  system  (prototype)  action,  and 
appropriately  modifies  the  actions  of  its  own  resources  to 
maintain  its  objectives. 

Given  the  above,  how  do  we  support  it?  We  require  a  high-level 
simulation  system  that  (1)  allows  high-level  manipulation  of 
objects  and  (2)  provides  high-fidelity  low-level  stimuli  to  the 
prototype  (and  which  translates  back  low-level  prototype  output 
into  appropriate  object  changes).  Such  a  system  can  be  partly 
based  on  now-available  commercial  knowledge-based  simulation 
systems,  e.g.  StmulCraft  (Carnegie  Group)  and  SimKit 
( I ntel I ICorp) .  These  systems  would  allow  specification  of 
scenario  strategy,  resources,  and  resource  Interactions,  and  then 
would  simulate  all  resource  activity.  However,  they  do  not 
address  item  (2)  above,  l.e.  the  mappings  to/from  the  prototype. 
What  is  needed  Is  a  "simulation  and  code  interface",  a  program 
capable  of  Interfacing  system  resource  simulations  and  the 
prototype,  providing:  (1)  input/output  linkages  between  the  two 
and  (2)  translation  of/to  resource  changes  to/from  concrete 
input/output  data. 

Under  this  approach,  the  simulation  system  controls  the  clock, 
allowing  Interactive  examination  of  partial  results  and 
interactive  editing  of  specific  variables  during  the  execution  of 
the  simulation. 

This  addresses  the  issue  of  scenario  generation  and  how  to  drive 
the  prototype. 
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(2)  PROBING  THE  PROTOTYPE  TO  DETERMINE  COVERAGE 

For  scenario  coverage  and  sensitivity  analysis,  a  probe  is 
required  into  the  prototype  that  outputs  not  only  the  data  that 
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results  from  the  simulated  stimulus  but  also  an  indication  of  the 
requirements  (and/or  solution  architecture)  in  terms  of  data 
structure,  processes  and  resource  cal  led  upon  to  generate  that 
output. 

(3)  GIVEN  SCENARIOS  THAT  CHARACTERIZE  EXPECTED  OUTPUT,  PROVE  THE 
SCENARIO  AND  DERIVE  COVERAGE 

Another  approach  to  the  coverage  problem  is  to  express  the 
solution  architecture  (or  possibly  even  the  requirements)  as  a  set 
of  PROLOG- 1  Ike  specifications,  treat  the  scenario  as  a  sequence  of 
Input/output  requirements,  and  statically  check  the  specifications 
to  see  how  they  can  be  combined  to  produce  the  specified  output 
from  the  Input.  The  specifications  used  to  prove  that  the  output 
can  be  generated  from  the  Input  will  indicate  the  coverage.  This 
would  extend  the  work  of  Shapiro  (Algorithmic  Program  Debugging). 
Note  that  a  prototype  is  not  necessary  to  determining  coverage 
under  this  approach,  a  solution  architecture  Is  sufficient. 

(4)  HOW  DOES  RPS  COMPARE? 

RPS.  Here  we  summarize  the  RPS  scenario  generation  and  results 
analysis  capabilities  and  the  premises  on  which  they  are  based: 

(1)  Scenarios  are  Initially  characterized  at  a  military  mission 
level  by  an  operations  expert.  (2)  Later,  a  funct icna I /process 
model  of  the  target  system  is  prepared  (by  another  analyst). 
Indicating  system  stimulus  types  and  the  sequence  of  functions 
they  activate.  The  model  also  ;nd!cates  the  resources  for  which 
the  functions  contend.  (3)  The  scenario  Is  Iteratively  decomposed 
until  it  consists  of  events  which  are  all  system  stimuli.  A 
scenario  I ibrary  and  some  generation  capabilities  help  relieve 
some  of  the  more  mechanical  aspects  of  this.  (4)  The  scenario  is 
run  against  a  performance  model  of  the  system.  (5)  During  the 
run,  the  performance  tool  collects  data,  computes  measures  of  how 
well  (time-wise)  the  modeled  system  performed,  and  at  the 
conclusion  of  the  run  presents  the  results  in  standard  statistical 
fashion.  (6)  Other  possible  feedback  Includes:  (for  some 
performance  models)  a  playback-capability  (graphical  reenactment 
of  each  event  and  the  transaction  It  triggered),  a  situational 
display  (displaying  the  viewable  aspects  of  the  scenario  as  it 
unfolds),  and  the  target  system’s  display  (what  the  target  system 
throws  out  on  screens  during  transactions  triggered  by  some 
scenario  event).  (7)  At  no  time  are  scenarios  run  against 
executab I e  code. 

In  conclusion,  RPS  scenarios  are  canned  and  the  prototype  (the 
system  performance  model)  generates  no  functional  output,  only: 

(1)  resource  utilization  data  for  the  performance  tool  to 
accumulate  and  (2)  displays.  The  motivation  behind  the  RPS 
approach  to  scenarios  Is  that  such  performance  data  is  invaluable. 

How  would  the  authors’  recommended  scenario  generation  approach 
extend  these  RPS  capabilities?  In  the  authors'  approach, 
scenarios  drive  more  than  just  performance  models,  they  drive 


wvw 

v 


m 


executable  models  which  generate  functional  output  that  can 
Interact  with  (and  be  related  to)  the  unfolding  scenario.  The 
result  Is  Improved  realism  In  scenario  and  target  system  model 
Interactions,  yielding  Insight  Into  how  an  antagonistic  force 
might  overcome  the  target  system  and  of  target  system 
countermeasures.  It  Is  Important  to  note  that  such  experiments 
will  suggest  revisions  In  more  than  Just  target  system 
performance;  functionality  and  control  Issues  will  have  to  be 
addressed  too.  By  combining  this  approach  and  the  capabilities 
RADC  Is  currently  developing  In  performance  models  (RPS,  to  a 
small  degree  the  Analyst)  and  executable  models  (VHLL  Prototyping 
tools),  the  result  Is  a  capability  to  evaluate  target  system 
function,  performance,  and  cost  trade-offs. 

Thus  for  maximum  effectiveness  of  the  adaptive  scenario  approach, 
the  authors  recommend  that  the  adaptive  scenario  effort  be  pursued 
so  that  the  resulting  scenarios  can  work  with  models  developed 
under  current  RADC  prototyping  efforts. 

RPS  will  provide  a  capability  to  represent  some  types  of 
requirements  and  solution  architecture  expressions  in  a  knowledge 
base  for  Interrogation  and  analysis  of  interactions.  This 
capability  cculd  be  the  basis  for  proving  scenarios  from  system 
specifications,  deriving  coverage. 

5-YEAR  OBJECTIVES: 

Provide  probes  for  prototypes. 

Provide  a  knowledge-based  simulation  baseline  system  that  can 
interface  and  run  against  performance  and  executable  models 
(especially  those  constructed  by  RPS,  VHLL  prototyping  tools,  and 
the  Analyst).  Build  a  configuration  manager  that  can 
sem i automat ica I ly  generate  the  si  mu  I  at  I  on/prototype  Interface 
(that  relates  high-level  sensor/actuator  changes  to  low-level 
prototype  Input/output)  based  on  available  sensor/ actuator  models 
and  user  requirements  on  prototype  level  of  detail  and  speed. 

10-YEAR  OBJECTIVES: 

Develop  a  system  to  statical ly  examine  the  design  of  the  prototype 
by  analyzing  the  Input/output  requirements  of  a  scenario  using  the 
design  specifications  as  a  proof  in  a  theorem-proving  system. 

Base  such  a  system  on  RPS  knowledge-based  requ Irements/model 
representat Ion  capabilities.  The  scenario  would  represent 
Instances  of  the  general  theorem  represented  by  the  design. 

RISK: 

The  knowledge-based  simulation  approach  to  scenario  generation  and 
driving  the  prototype  involves  little  risk,  given  the  commercial 
availability  of  wnat  wouid  be  a  key  component  of  such  a  system. 
Both  scenario  coverage  approaches  invite  risk  because  tracking 
what  requirements  were  exercised  by  a  scenario  presents 
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Integration  and  technical  risks.  Assuming  Integration  risks  are 
controlled,  the  likelihood  of  results  Is  60  percent  for  prototype 
probes  and  40  percent  for  the  static  analysis  (theorem  proving). 


COST: 


Knowledge-based  simulation 

Combine  simulation  with  code 

Provide  Intelligent  configuration  manager 


2  man-years 
2  man-years 
2  man-years 


Prototype  probe  (dependent  on  the  prototyping  approaches) 
Create  static  scenario  coverage  analysis  system  8  man-years 


CONTINGENCIES: 


Part  of  both  the  scenario  generation  and  scenario  coverage 
research  (especially  probing)  depends  heavily  on  the  approaches 
taken  In  the  prototyping  research.  Also  dependent  on  the  testbed 
Integration  effort. 

CONCLUSIONS: 

This  effort  should  be  funded.  Recommend  that  scenarios  developed 
under  the  scenario  generation  effort  be  able  to  work  with  all 
prototyping  tools.  Recommend  static  scenario  coverage  analysis 
effort  be  a  possible  follow-up  to  the  RPS  effort. 

AUTHOR  NAME:  Stephen  Sherman,  Michael  Konrad 
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RET  R&D  Effort 


ISSUE  NAME:  Scenario  Generation  Support  &  Scenario  Coverage  Analysis 

TRACK:  Evolutionary 

OBJECTIVE: 

Produce  a  knowledge-based  simulation  tool,  a  dynamic  coverage  tool 
(a  prototype  probe),  and  a  static  coverage  tool  (proves  scenario 
from  system  specification). 


SCOPE: 


The  scenario  generation  effort  will  not  attempt  to  pioneer 
knowledge-based  simulation.  The  two  coverage  tools  should  be 
pursued  In  two  parallel  R&D  efforts.  The  static  coverage  tool 
effort  will  attempt  to  pioneer  an  application  of  existing 
technology. 

TECHNICAL  APPROACH: 

Working  closely  with  the  prototype  development  efforts,  provide  a 
knowledge-based  simulation  system  for  scenario  generation.  The 
system  must  contain  a  representation  of  system/scenario  resources 
and  the  scenario  strategy.  The  system  must  be  extended  to  Include 
a  "simulation  and  code  interface"  which  translates  between  high- 
level  system  resource  changes  and  prototype  inputs  and  outputs. 

For  static  coverage  analysis,  the  prototype  design  (solution 
architecture)  Is  represented  as  a  set  of  PROLOG-like  predicates. 
The  scenario  Is  represented  as  sequence  of  Input  values  and  output 
responses.  A  theorem  prover  will  select  the  predicates  that 
accept  the  specified  input  and  determine  what  helped  produce  the 
specified  output.  The  predicates  that  are  required  to  derive  the 
output  from  the  input  are  a  measure  of  the  coverage.  The  design 
predicates  must  then  be  related  to  requirements  to  Indicate 
requirements  coverage.  The  RPS  Knowledge-based  system 
"Translator"  can  serve  as  a  basis  for  this  effort.  The  Translator 
is  capable  of  taking  system/operator  resource  models  as  input  and 
generating  PROLOG-specI f Ications. 

BACKGROUND: 

The  key  tools  are  knowledge-based  simulation,  knowledge-based 
analysis  and  theorem  proving. 

The  concept  of  knowledge-based  simulation  was  defined  by  Reddy  and 
Fox  of  Carneg le-Mel Ion,  "KBS:  An  Artificial  Intelligence  Approach 
to  Flexible  Simulation",  and  also  implemented  In  the  ROSS 
simulation  system  developed  by  the  Rand  Corporation,  "ROSS:  An 
Object-Oriented  Language  for  Constructing  Simulations." 
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REFERENCES: 


1.  KBS:  An  Artificial  Intelligence  Approach  to  Flexible 
Simulation,  Y.V. Reddy  and  Mark  S.  Fox,  CUM-RI -TR-82-1 , 
Robotics  Institute,  Carneg ie-Mel Ion  University,  Pittsburgh, 
Pennsylvania,  15213,  14  September  192. 

2.  ROSS:  An  Object-Oriented  Language  to  Constructing 
Simulations,  David  McArthur,  Philip  Klahr,  Sanjai  Narafn,  R 
3160-Af,  December  1984. 

3.  Algorithmic  Program  Debugging,  Ehud  Y.  Shapiro,  The  MIT 
Press,  Cambridge,  Massachusetts,  London,  England. 

4.  SimulCraft  is  a  commercial  product  of  the  Carnegie  Group. 

5.  SimKit  is  a  commercial  product  of  IntelliCorp. 

DURATION: 


Scenario  Generation  (SG)  3  Years 

Static  Scenario  Analyzer  (SSA)  4  Years 

COST: 

SG  $  .9  ml  I  I  Ion 

SSA  $1.2  mill  ion 

DELIVERABLE:  Product  Date 


SG  Knowledge-Based  Simulation 

SG  Simulation  and  Code  Interface 

SG  Configuration  Manager-Intelligent 


12  months 
24  months 
36  months 


SSA  Enhance  RPS  representation  of  designs 
SSA  Input/Output  Representations 
SSA  Theorem  Proving  Techniques 
SSA  Design-Requirements  Connection 


18  months 
24  months 
40  months 
48  months 


Evolutionary  Track 


PROBLEM  DESCRIPTION: 

There  are  two  problems:  (1)  Make  sure  that  the  prototype  and 
scenario  are  consistent,  complete  and  logically  correct  In  a 
static  check.  (2)  Determine  the  validity  of  the  results  of 
executing  a  scenario  on  the  prototype  In  a  dynamic  check. 

VALUE  IN  REQUIREMENTS  PROCESS: 

The  credibility  of  the  RET  depends  on  (1)  our  ability  to  associate 
requirements  with  resource  requirements  and  (2)  our  ability  to 
accurately  predict  the  resource  requirements.  Thus  validation  of 
the  prototype  and  scenario  increases  confidence  In  the 
predictions. 

SOLUTION  APPROACH: 

Provide  a  knowledge-based  syntax  and  semantics  checker  for  a 
static  check.  It  is  assumed  that  the  prototype  and  scenario  have 
a  formal  language  description.  The  knowledge  base  will  also 
contain  meta-knowledge  about  the  domain  model.  The  checker  will 
examine  the  descriptions  for  the  prototype  and  scenario  for 
consistency  and  completeness. 

For  a  dynamic  check,  the  prototype  results  must  be  compared  with 
real  implementations  using  the  same  scenario.  The  difficulty  is 
in  identifying  good  comparison  metrics  and  In  metering  both  the 
prototype  and  Implemented  system.  Unfortunately  the  results  are 
only  useful  after  decisions  based  on  requirements  and  prototype 
accuracy  have  been  made.  In  order  to  use  Information  on  accuracy, 
we  need  a  library  of  validated  correct  (within  limits)  prototypes. 
New  prototypes  need  to  be  compared  to  validated  prototypes  through 
reusability  techniques.  If  the  differences  between  the  new 
prototypes  and  the  validated  prototypes  are  small,  confidence  In 
the  accuracy  Is  improved.  The  key  to  this  approach  Is  the 
creation  of  a  library  of  validated  prototypes  or  parts  of 
prototypes  and  techniques  for  retrieving  Information  from  that 
I ibrary. 

5  YEAR  OBJECTIVES: 
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Start  a  library  of  validated  prototypes  and  begin  developing 
retrieval  techniques.  Define  a  formal  language  for  describing 
prototypes  and  scenarios. 

10  YEAR  OBJECTIVES: 

Continue  developing  the  reusable  prototype  system  Including 
breaking  down  the  prototype  Into  reusable,  validated  prototype 
parts.  Develop  the  techniques  to  validate  a  prototype  and 
scenario  for  consistency  and  completeness. 


RISK: 


The  static  check  Is  higher  risk  due  to  the  difficulty  of 
coordinating  languages  for  scenarios  and  prototypes.  However,  if 
that  problem  Is  solved,  the  techniques  for  determining  consistency 
and  completeness  being  developed  for  expert  systems  could  be 
transferred  to  the  domain  of  prototypes  and  scenarios. 

The  risk  for  the  dynamic  check  Is  low  since  reusability  Is  a  large 
research  area  now  and  val I  dating  our  prototypes  must  be  done. 


COST: 


The  static  check  and  language  definition  Is  a  6  man  year  effort. 
The  dynamic  check  Is  a  continuous  effort  In  building  a  library  of 
validated  prototypes.  The  effort  to  convert  a  reusable 
programming  technique  to  prototypes  Is  a  3  man  year  effort. 


CONTINGENCIES: 


Walt  for  reusability  technology  to  be  developed.  Coordinate 
scenario  and  prototype  formal  language  descriptions. 


CONCLUSION: 


Fund  the  library  creation  of  metered  and  validated  prototypes. 
Examine  a  formal  language  approach  for  both  the  prototype  and 
scenario. 


AUTHOR  NAME: 


Stephen  Sherman 


.  Evolutionary  Track. 


PROBLEM  DESCRIPTION: 

Driving  a  prototype  against  a  scenario  can  produce  considerable 
data  which  must  be  collected  for  presentation  and  analysis. 
Presentation  can  occur  both  during  and  after  the  experiment.  An 
analysis  capability  Is  needed  to  evaluate  the  combined  results  of 
many  experiments. 

VALUE  IN  REQUIREMENTS  ENGINEERING  PROCESS: 

Providing  tools  to  Interpret  results  of  prototype  experiments  Is 
critical  to  evaluation  of  prototypes,  and  therefore  of 
requ I rements. 

SOLUTION  APPROACH: 

Two  sets  of  needs  are  to  be  addressed,  those  of  a  prototype 
experimenter  and  those  of  a  mission  user.  The  mission  user 
collaborates  with  the  experimenter  In  establishing  the  scope  and 
objectives  of  an  experimental  run(s)  and  In  selecting  the  right 
scenarlo(s)  for  the  run(s).  The  prototype  experimenter  sets/I  inks 
up  the  prototype  and  scenario  for  the  experimental  run(s).  During 
scenario  execution,  the  experimenter  may  need  feedback  on  how  the 
prototype  Is  doing  (e.g.  in  utilization  of  resources).  The 
mission  user  must  have  feedback  In  order  to  assess  how  the 
prototype  performs  against  the  scenario.  The  experimenter  needs 
to  record/capture  comments  elicited  from  the  mission  user  during 
the  experiment.  After  the  experiment,  both  the  mission  user  and 
experimenter  might  want  to  analyze  results  of  the  current  run, 
possibly  In  the  context  of  pt  ev'ous  runs. 

Thus  the  following  capabilities  should  be  provided: 

(1)  Visual  presentation  and  analysis  of:  the  unfolding  scenario 

situation,  what  outputs  (displays,  functional  responses  to 
the  scenario)  the  prototype  generates,  and  what  resources 
the  prototype  utilizes.  The  data  presentation  format  (e.g. 
of  scenario  situational  displays  vs.  prototype  displays,  or 
what  plots  against  what)  should  be  flexible. 

(2)  To  (1)  above  should  be  added  an  aggregate  analys.ls 

capability.  The  experimenter  and  mission  user  can  specify 
what  kinds  of  aggregate  analyses  they  want  on  data  generated 
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during  the  run,  for  viewing  both  during  the  run  and  after. 
Examples  of  aggregate  analyses  Include:  number  of  resources 
left  (e.g.  planes),  mean  response  time,  and  percentage  of 
hits  (e.g.  simulated  hits  on  attacking  planes).  For 
presentation  of  aggregate  analyses  results,  standard 
statistical  presentation  formats  should  be  available. 

(3)  A  capability  for  historical  analyses  of  results.  For  example 

to  do  a  sensitivity  analysis,  one  might  want  to  do  a  number 
of  experimental  runs  In  which  a  scenario  Is  fixed  for  a  time 
and  run  against  different  prototypes  and/or  vice  versa. 

(4)  A  capability  to  capture  mission  user  responses  during/after 

the  experimental  runs.  This  might  Involve  no  more  than 
careful  recording  of  what  goes  on  during  each  experimental 
run.  An  additional  approach  would  have  the  user  temporarily 
pause  the  experimental  run  to  make  "debugging  changes"  or 
textual  annotations  to  the  unfolding  events  of  what  should 
have  been  the  prototype's  response/display/performance. 

(5)  A  capability  to  formally/informally  compare  expected 

prototype  outputs  (which  are  part  of  the  scenario's 
definition)  against  scenario  events.  The  formal  comparison 
approach  Is  addressed  by  the  static  scenario  analysts  tool 
recommended  as  a  part  of  the  "Scenario  Generation  Support  & 
Scenario  Coverage  Analysis"  research  Issue.  For  the  near-, 
term,  the  comparison  will  be  done  by  humans  with  the 
assistance  of  data-management  tools. 

5-YEAR  OBJECTIVES: 

A  central  repository  for  the  collection  of  all  data  arising  from 
prototype  experimental  runs.  Database  management  facilities  (e.g. 
browsing)  available  to  aid  human  analysis  of  data. 

Specific  analyses  (especially  analyses  during  experimental  runs) 
remain  under  the  control  of  the  different  prototyping  tools  until 
they  can  be  Integrated. 

10-YEAR  OBJECTIVES: 

From  a  single  experimental  run,  get  a  full  range  of  analyses  (not 
Just  that  provided  by  a  single  tool  on  a  single  model)  and  In 
which  all  data  can  be  cross-correl ated  (even  during  the  run). 

Knowledge-based  aids  for  evaluation  of  prototype  sensitivity. 
Analyses  providing  results  based  on  combined  experimental 
Information. 

A  tool  that  Indicates  which  experimental  runs  should  be  performed 
next  based  on  an  evaluation  of  system  requirements  that  need 
further  testing  or  stressing. 

RISK: 
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Near-term  objectives: 
to  high  risk. 


Long-term  objectives  Involve  moderate 


Knowledge-based  system  for  analysis 
of  experiment  results. 


6  man-years. 


CONTINGENCIES: 


A  continuing  R&D  effort  In  prototyping.  Reasonably  successful 
Integration  of  the  RET,  especially  In  the  area  of  the  database. 
Success  In  the  ’'adaptive  scenario  generation"  approach  of  the 
"Scenario  Generation  Support  &  Scenario  Coverage  Analysis"  RET 
research  Issue. 


CONCLUSION: 


Recommend  funding.  It  Is  crucial  to  the  RET.  No  special  funding 
Is  needed  for  the  near  term,  provided  that  (1)  the  "Database"  RET 
development  effort  provides  the  appropriate  data  management 
facilities  and  (2)  all  prototyping  tools  provide  the  appropriate 
capabilities  for  the  analysis  of  scenar io-prototype  execution 
resu I ts. 


AUTHOR  NAME: 


Stephen  Sherman,  Michael  Konrad. 
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RET  R&D  Effort 

ISSUE  NAME:  Scenario  Execution  and  Analysis  of  Results 
OBJECTIVE: 

Provide:  visual  presentation  and  analysis,  aggregate  analysis, 
and  historical  analysis  of  results  from  driving  a  prototype 
against  a  scenario. 


SCOPE: 


Research  addresses  the  following  subjects:  statistical  analysis 
of  data,  control  of  prototype  experiments,  sensitivity  analysis, 
and  experiment  design. 

TECHNICAL  APPROACH: 

Near-term  approach: 

RPS.  In  the  "Scenario  Generation  Support  &  Scenario  Coverage 
Analysis"  research  issue,  the  capabilities  of  RPS'  performance 
modeling  tools  is  discussed.  These  tools  work  off  performance 
models,  collect  performance  data,  and  provide  relevant  analyses  on 
the  results.  Thus  RPS  provides  significant  capabilities  in  visual 
presentation  and  analysis  (Items  (1)  and  (2)  of  the  Solution 
Approach)  for  performance  data. 

RET  database.  The  RET  database  should  serve  as  the  col  lector  and 
repository  for  all  information  generated  during  prototype  runs. 
This  would  help,  for  example,  compare  results  of  running  the  same 
scenario  against  both  a  performance  model  (RPS)  and  executable 
model  (VHLL  Prototyping  tools)  as  a  basis  for  a  trade-off 
analysis.  The  RET  database  should  provide  data  management  support 
for  human  analysis.  These  requirements  on  the  RET  database  are 
consistent  with  those  described  under  the  "RET  Database  Management 
System"  R&D  effort  description. 

However,  there  is  still  a  need  to  provide  specific  analyses  when 
scenarios  are  run  against  specific  target  system  models,  e.g.  the 
different  analyses  that  are  performed  on  the  different  performance 
models  one  can  construct  with  RPS  tools.  For  the  near  term,  these 
analyses  will  probably  remain  under  control  of  the  tool  that 
helped  construct  the  model /prototype.  With  testbed  Integration, 
the  experimenter  should  be  able  to  orchestrate  the  running  of  a 
scenario  against  several  models  (say,  of  the  same  solution 
architecture)  and  combine  analyses. 

Long-term  approach: 

Expand  the  adaptive  scenario  generation  capability  discussed  in 
the  "Scenario  Generation  Support  &  Scenario  Coverage  Analysis"  RET 
research  issue  so  that  the  simulation  knowledge  base  can  be  used 
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to  evaluate  prototype  sensitivity,  e.g.  between  different 
scenarios  and  different  prototypes.  Analyses  should  provide 
results  based  on  combined  experimental  Information. 

Assuming  that  requ Irements,  scenarios,  and  prototypes  are 
maintained  In  the  same  knowledge  base  (of  which  the  simulation 
knowledge  base  constitutes  a  part),  another  capability  can  be 
identified  for  the  very  long  term.  From  an  aggregate  analysis  of 
sens  It  I v Ity /coverage  analyses  results,  an  Indication  will  be 
provided  of  what  future  experimental  runs  should  be  performed 
based  on  an  evaluation  of  system  requirements  that  need  further 
testing  or  stressing. 

BACKGROUND: 

The  authors  know  of  no  mature  efforts  on  analyzing  results  from 
prototype  runs,  though  there  are  analogs  in  testing  technology. 
However,  CO  examines  the  overall  software  prototyping  approach. 
Investigates  the  experimenter-mission  user  relationship,  and  makes 
some  practical  suggestions  of  how  to  make  the  prototyping 
experiment  effective. 

REFERENCES: 

Oj  Alavi,  Maryam,  "An  Assessment  of  the  Prototyping  Approach  to 
Information  Systems  Development",  Communications  of  the  ACM,  Vol. 
27,  No.  6,  June  1984. 

DURATION; 

Knowledge-based  system  for  analysis  3  years 

of  experiment  results  (AER) 

COST  (in  S): 

AER  $  .9  mill  ion 

DELIVERABLES:  Product  Date  (months  after  start): 

Enhance  simulation  knowledge  base/RET  database 
for  Historical  analyses 
Prototype  sensitivity  analysis  tool 
Experiment  advisor  tool 


16  months 
24  months 
36  months 


.  Evolutionary  Track 


PROBLEM  DESCRIPTION: 

The  uncertainty  of  software  costs  Is  one  of  the  major  cost  Issues. 
The  existing  cost  models  have  several  subjective  parameters.  The 
manager  dealing  with  these  things  needs  tools  for  more  reliable 
estimations  of  cost,  time  etc.  to  aid  making  scheduling  and 
resource  allocation  decisions. 

VALUE  IN  REQUIREMENT  PROCESS: 

Better  estimates  of  cost,  time  and  performance  can  help  management 
in  choosing  between  alternate  solutions.  This  also  facilitates 
the  examination  of  functionality  vs.  cost  trade-off  (also  known  as 
Impact  analysis  or  sensitivity  analysis)  for  better  project 
management. 

SURVEY  OF  CURRENT  TECHNIQUES: 

Cause-and-ef f ect  relationships  as  a  basis  for  software  cost 
analysis  and  estimation  have  been  investigated  by  various  people 
for  the  last  twenty  years.  Most  of  the  models  use  the  size  of  the 
project  to  estimate  the  man-months  required  to  complete  the 
project,  as  this  factor  seems  to  correlate  best  with  the  cost. 
These  models  are  based  on  data  from  past  projects.  Sometimes  the 
data  Is  translated  to  charts  and  graphs.  Another  approach  is  to 
formulate  a  parametric  model,  involving  mathematical  functions  of 
several  variables,  suggested  by  previous  experimentation  and 
engineering  judgment.  Statistical  techniques  are  used  to 
determine  the  relevance  of  the  set  of  variables  used  in  the 
equation;  constants  of  the  equation  (parameter  estimation)  are 
based  on  historical  data. 

CWolverton  75j  has  classified  these  efforts  Into  five  categories  - 
-  Top-down  estimation.  Bottom-up  estimation,  similarities  and 
differences  estimation,  ratio  estimation  and  standard  estimation. 
CBoehm  81j  has  a  slightly  different  classification  of  these 
approaches  —  based  on  algorithmic  modeling,  expert  Judgment, 
estimation  by  analogy,  pricing  to  win  (cost  =  f(what  customer  can 
pay))  and  Parkinson's  law  ("The  project  expands  to  consume  the 
budget  available").  Both  of  them  observe  that  two  of  these  models 
should  be  used  for  better  reliability. 


Besides  estimating  the  total  cost  of  development  of  the  entire 
system.  It  is  useful  to  have  separate  estimates  for  each  phase  of 
the  life  cycle  of  the  system.  Similarly  estimates  for  each  module 
and  unit  may  be  needed  to  optimize  the  schedule  and  cost  against 
deadlines  penalties. 

Past  success  stories  Include  Wolverton's  studies  (of  1974)  and  the 
RCA  PRICE  S  system  [Friedman  793.  Recent  success  stories  Include 
the  Walston  and  Felix  studies  (IBM)  and  C0C0M0  model  [Boehm  813- 

The  reliability  research  has  two  distinct  flavors.  The  first  one 
Is  oriented  towards  tools  for  static  code  analysis,  test  case 
generation,  symbolic  execution,  correctness  proofs  etc.  The  other 
Is  nearer  to  the  statistical  modeling  approach  and  leads  to 
reliability  models,  methodology  for  code  Inspection,  standard  test 
procedures  etc.  The  models  for  measurement,  estimation  and 
prediction  are  based  on  a  variety  of  estimation  statistics 
(Rayleigh,  Bayslan,  Markov,  Geometric  etc)  for  failure  rates/error 
distribution.  Recent  methods  of  fault  handling  by  fault-tree, 
event-tree,  and  Influence-tree  analysis  also  seem  to  be  promising 
methods.  The  Spiral  model  of  the  software  life  cycle  has  risk 
analysis  In  every  phase.  Success  stories  Include  IBM  Clean  Room 
and  Bel  I  Labs  ESS. 

The  modeling  approach  has  Its  own  limitations.  Data  from  old 
projects  may  not  be  useful  for  estimating  the  cost,  risk  etc.  of  a 
new  project.  Rapidly  evolving  technology  (better  programming 
environments,  richer  set  of  tools,  automatic  programming, 
workstations)  have  changed  the  productivity  of  programmers.  The 
models  can  not  take  many  factors  (such  as  implementation 
constraints,  management  techniques,  programmer  qualification)  into 
account.  Solutions  based  on  concurrent  programs  and  distributed 
systems  pose  problem  for  these  models.  The  models  should  be 
supplemented  by  other  Informal  means  of  estimation  of  cost,  time 
and  risk  of  development. 

SOLUTION  APPROACH  (1):  METRICS 

A  metric  is  a  measure  of  some  character Istlcs  of  a  software 
system.  In  the  Metrics  Guided  Methodology  [RAM  853,  metrics  are 
used  as  a  too!  for  software  development.  The  metrics  provide 
feedback  to  the  developer,  letting  him  know  his  progress.  It  can 
also  be  used  predict  where  the  project  Is  going  by  estimating 
future  size  and  cost,  or  It  may  Indicate  that  the  current  design 
Is  too  complicated  and  unstructured.  It  can  be  used  through  the 
software  life  cycle  from  predictions/estimations  about  new 
products  to  evaluation/  maintenance  of  existing  products. 

The  Metrics  Guided  Methodology  (MGM)  helps  get  better  estimates  of 
the  cost,  time  and  performance  of  the  system  being  designed. 
Requirements  metrics  can  indicate  the  complexity  of  designs  and 
implementations  at  an  early  stage  and  suggest  simplifications  In 
particular  areas  or  allocation  of  more  resources  In  those  areas. 


These  can  help  In  making  Intelligent  compromises  between  target 
system  performance,  project  deadlines  and  allocation  of  manpower 
to  the  project  CRAM  85]. 


The  use  of  metrics  for  prediction  of  system  qua! Ity/dlff iculty  of 
design  etc.  can  save  a  lot  of  effort  and  Investment  In 
prototyping.  The  metrics  substitute  partfafly  for  prototyping  of 
the  system.  "Feedback  metrics"  give  immediate  feedback  to  the 
designer  and  hence  he  does  not  have  to  wait  until  later  design 
stages  to  detect  problems.  This  leads  to  smaller  and  fewer  design 
iterations  for  the  same  quality.  The  measurement  and 
interpretation  of  metrics  can  be  automated  by  expert  system 
technology. 

When  using  MGM  for  a  specific  project,  metrics  should  be  collected 
based  on  the  objectives  we  want  to  achieve  in  that  project.  This 
view  has  been  expressed  by  several  people  [BAS  84],  This  requires 
one  to  obtain  the  objectives  (e.g.  system  availability, 
performance)  from  the  requirements  and  then  develop/select  metrics 
for  those  objectives.  Development  of  metrics  Is  mostly  subjective 
at  present,  though  limited  validation  can  be  done  by  experimental 
means.  Using  prior  data  and  carefully  documented  results  from 
past  analysis  on  prototypes  or  similar  projects  can  also  be  used 
to  boost  confidence  in  particular  metrics  chosen  for  the  project. 

BACKGROUND: 

The  term  "metrics"  have  been  traditionally  used  to  denote 
complexity  measures.  Metrics  have  also  been  used  for  maintenance 
of  large  software  systems.  The  term  metrics,  in  general,  refers 
to  all  kinds  of  measures  of  various  characteristics  of  software. 
The  particular  characteristics  may  be  useful  for  assessment  and 
estimation  of  the  quality  of  requirements  and  prediction  of 
difficulties  in  later  stages  of  software  lifecycle.  Other 
measures  include  the  use  of  techniques  like  utility  functions  and 
risk  analysis  in  making  decisions  during  software  development. 

Metrics  are  used  to  evaluate  software  process  and  product.  They 
are  also  used  as  a  tool  for  software  development.  They  can  be 
used  to  moni+or  the  stability  and  quality  of  existing  systems. 
There  are  several  classifications  of  metrics  [Basil!].  Metrics 
can  be  divided  into  product  metrics  (number  of  decisions  and 
interfaces)  and  process  metrics  (based  on  time  of  development, 
number  of  errors).  A  second  dichotomy  Is  quality  metrics  (which 
would  evaluate  the  product  as  good  or  bad  relative  to  a  specific 
model)  vs.  Invariant  metrics  (which  are  Independent  of  environment 
and  product  except  for  the  effects  of  size  on  cost).  A  third 
classification  would  distinguish  between  a  priori  metrics  and  a 
posterior  metrics.  An  a  priori  metric  is  used  to  estimate  and 
evaluate  the  product  being  designed.  An  a  posterior  metric  is  a 
measure  of  the  existing  product  after  the  design  is  complete. 

It  is  suggested  that  a  single  metric  Is  not  sufficient  for  large 
software  projects  and  hence  a  spectrum  of  metrics  should  be  used 
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CRAM  853.  Metrics  to  be  used  should  be  selected  based  on  the 
objectives  of  a  particular  project.  There  Is  a  need  for 
experimental  validation  of  metrics,  but  little  validation  can  be 
expected  because  of  the  difficulty  In  conducting  controlled 
experiments  In  this  area. 

RESEARCH  ISSUES: 

There  Is  a  need  to  define  and  validate  a  set  of  metrics  to 
estimate  the  cost,  risk  and  time  required  In  each  phase  of 
development  of  a  software  system.  A  typical  validation  process 
can  be  used  CBoehm  783,  CMcCal I  77].  The  selection  of  metrics  Is 
subjective  and  based  on  experience.  Then  one  clusters  the  metrics 
by  factor  analysis.  This  Is  followed  by  collection  of  data  and 
regression  analysis.  The  metric  scoring  method  Is  an  Important 
issue,  and  the  method  should  be  defined  properly  and  then 
automated. 

The  ’’size”  of  the  project  seems  to  a  major  factor  In  determining 
the  cost  of  the  project.  Current  metrics  used  In  size  estimation 
(SLOC,  number  of  modules  etc)  are  not  very  satisfactory.  It 
relies  heavily  on  the  ability  of  the  software  engineer  in-charge 
to  estimate  the  size  of  software  product  based  on  the 
requirements,  which  may  be  Imprecise  or  subject  to  change.  There 
is  a  need  for  tools  to  estimate  the  size  of  the  system  from  the 
requirements,  using  knowledge  about  past  projects  and  current 
technology.  Similarly,  metrics  that  represent  project  size  (SLOC, 
number  of  modules  etc)  are  not  satisfactory  for  many  purposes. 
There  is  a  need  for  better  metrics  to  represent  the  magnitude  of 
the  project. 

The  collection  of  metric  data  from  old  projects  can  be  used  to 
calibrate  estimation  models  so  that  these  models  can  be  applied  to 
new  projects.  Similarly,  design  and  validation  of  metrics  that 
measure  the  effects  of  programming  environments,  project 
constraints  etc.  would  be  useful.  There  Is  definitely  a  need  for 
developing  methods  for  estimating  the  cost,  time  and  risk  Involved 
In  developing  distributed  systems,  concurrent  programs,  knowledge 
based  systems  etc.  A  methodology  to  derive  a  good  set  of  metrics 
for  these  purposes  has  been  discussed  In  CBoehm  783  and  CMcCal I 
773. 
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COST: 


(1)  Metric  Guided  Methodology 
metrics  for  cost,  time,  risk; 
project  size  metrics. 
Measurement/ana lysis  tools 

(2)  Add  Metrics  for  distributed 
systems,  etc. 

(3)  Tools  for  fault-handling 
analysis  and  reliability 
estimation 

DELIVERABLE  PR00UCT: 


3  person  yrs  .45  million 


2  person  yrs  .3  million 


3  person  yrs  .45  million 


1.  Metrics  for  estimation  of  "size"  of  the  projects. 

2.  Methodology  for  estimation  of  cost,  time  and  risk  of  a 
project.  This  will  combine  rapid  prototyping  and  predictive 
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PROBLEM  DESCRIPTION: 

Requirements  are  Informal  statements  of  need.  Requirements  bound 
the  space  of  possible  specifications.  They  should  be  internally 
consistent  and  feasible  within  the  state  of  the  art.  The 
requirements  affect  the  rest  of  the  design  process  tremendously 
and  any  mistake,  missing  Information,  or  Inconsistency  may  be  very 
difficult  to  correct  at  a  later  stage  of  design.  Similarly  the 
quality  of  requirements  affect  the  complexity  of  work  in  later 
stages.  Hence  It  Is  Important  to  create  quality  requirements  and 
there  should  be  tools  to  measure  the  quality  of  the  requirements. 

VALUE  IN  REQUIREMENT  PROCESS: 

Requirements  evaluation  will  help  one  to  choose  between  two 
alternate  sets  of  requirements  for  the  same  system.  Evaluation 
can  expose  statements  that  complicate  later  design  stages  and 
suggest  restructuring  to  improve  them.  Measurements  of  ease  of 
modifiability  will  help  one  to  make  the  requirements  more 
maintainab I e. 

Indications  of  when  to  terminate  iterative  requirements 
engineering  activities  are  needed.  Though  it  Is  difficult  to 
measure,  some  estimate  of  sensitivity  to  missing  knowledge  Is 
required  if  one  Is  to  have  confidence  in  the  system. 

SURVEY  OF  PAST  APPROACHES: 

Requirements  evaluation  Is  a  relatively  new  area.  Requirements 
are  often  Informal  and  evaluated  by  Inspection.  The  criteria  for 
evaluation  are  still  evolving.  Henlnger  (1980)  suggested  that: 
requirements  should  be  modifiable,  used  as  reference  for 
maintenance,  reflect  forethought  about  the  life  cycle  of  the 
system,  and  characterize  acceptable  response  to  undeslred  events. 
He  also  suggested  a  format  for  the  requirement  definition 
document. 

The  first  step  In  requirements  definition  Is  to  produce  a 
conceptual  model  of  the  system.  There  exists  many  different  kinds 
of  formal  models  whose  use  reduces  ambiguity  and  vagueness  In  the 
requirements.  Objectives  models  help  to  hierarchically  analyze 
and  describe  customer  goals  In  graphic  or  text  form.  Conceptual 
data  models  help  In  analysis  of  major  data  and  their 


relationships,  helping  the  capture  and  structuring  of  Information 
needs.  Conceptual  process  models  help  In  the  analysis  of  those 
target  system  processes  Identified  In  the  objectives  model,  and 
also  help  analyze  process-process  and  process-environment 
Interactions.  Data  flow  and  control  flow  models  help  to  describe 
the  behavior  of  the  system  in  more  detail  [Miyamoto,  Yeh]. 

The  requirements  are  sometimes  expressed  In  natural  language  or 
using  graphic  symbols.  Some  work  has  been  done  to  structure  and 
format  natural  language,  without  Imposing  rigorous  syntax  or 
semantics.  Formal  languages  are  better  suited  to  requirements 
evaluation,  but  the  use  of  formal  languages  for  stating 
requirements  has  not  lead  to  any  great  success.  Natural  language 
expressions  of  requirements  are  difficult  to  check  for 
completeness  and  consistency.  It  is  also  difficult  to  partition 
such  expressions  Into  different  types  (e.g.  functional,  non¬ 
functional  requirements),  unless  the  specifier  was  very  careful. 
They  tend  to  be  ambiguous,  unclear  and  Inconsistent.  Most 
practical  systems  fall  Into  the  second  category:  structured 
natural  language  representations.  These  include  PSL/PSA,  SADT  and 
RSL  [see  references].  There  have  also  been  attempts  to  use  an 
Ada-like  notation,  but  their  use  often  leads  to  expression  of  many 
low -level  details,  unnecessarily  limiting  the  freedom  of  designers 
later  In  development. 

Requirements  are  often  divided  into  two  types  —  functional  and 
non-functional .  Functional  requirements  Identify  the  system 
services  wanted  by  the  user.  Often  these  don't  relate  (are 
orthogonal)  to  the  Implementation.  In  principle,  the  functional 
requirements  of  a  system  should  be  both  complete  and  consistent. 
Completeness  means  that  all  services  wanted  by  the  users  should  be 
specified.  Consistent  means  that  no  two  requirements  should 
contradict  each  other.  The  non-functional  requirements  Include 
constraints  on  the  Implementation:  response  time,  time  of 
completion,  ccmpat ib 1 1 i ty  with  existing  software  and  hardware, 
etc.  These  often  change  as  technology  changes.  They  often  tend 
to  conflict  with  the  functional  requirements  and  Induce  tradeoffs 
in  the  design.  The  non-funct Iona  I  requirements  are  generally 
expressed  In  natural  language. 

The  purpose  of  requirement  validation  Is  to  check  for  the 
consistency,  completeness  and  feasibility  of  requirements.  One 
tool  to  assist  the  human  validator  would  retrieve  the  set  of  a  I  I 
requirements  referencing  a  common  function  of  the  system. 
Simulation  is  often  used  to  show  the  feasibility  and  completeness 
of  the  requirements.  Simulation  can  be  often  very  costly  and  time 
consuming.  Furthermore  It  Is  difficult  to  change  the  simulator, 
as  the  requirements  change  and  evolve  [Vick,  Davis  19771.  Rapid 
prototyping  has  been  tried.  Rapid  prototyping  can  be  accomplished 
thru  use  of  high-level  languages,  libraries  of  utilities  (c-shell 
in  Unix),  and/or  by  reducing  the  error-handling  and  quality  of  the 
user  interface. 

SOLUTION  APPROACH  (1):  METRICS 
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A  metric  Is  a  measure  of  some  characteristics  of  a  software 
system.  In  the  Metrics  Guided  Methodology  [Ram  85],  metrics  are 
used  as  a  tool  for  software  development.  Metrics  provide  feedback 
to  the  developer,  letting  him  know  his  progress.  It  can  also  be 
used  to  predict  where  the  project  Is  going  by  estimating  future 
size  and  cost,  or  It  may  Indicate  that  the  current  design  Is  too 
complicated  and  unstructured.  It  can  be  used  throughout  the 
software  life  cycle  from  predictions/estimations  about  new 
products  to  evaluation/maintenance  of  existing  products.  For 
example,  the  Spiral  model  of  the  software  life  cycle  uses  risk 
analysis  at  every  stage.  Metrics  can  be  used  In  every  stage  to 
estimate  the  risk  factors. 

The  Metrics  Guided  Methodology  (MGM)  helps  get  better  estimates  of 
the  cost,  time  and  performance  of  the  system  being  designed. 
Requirements  metrics  can  Indicate  the  complexity  of  designs  and 
Implementations  at  an  early  stage,  and  suggest  simplifications  In 
particular  areas  or  allocation  of  more  resources  In  those  areas. 
These  can  help  In  making  Intelligent  compromises  between  target 
system  performance,  project  deadlines,  and  allocation  of  manpower 
to  the  project  [RAM  85]. 

BACKGROUND: 

Metrics  are  used  to  evaluate  software  process  and  product.  They 
are  also  used  as  a  tool  for  software  development.  They  can  be 
used  to  monitor  the  stability  and  quality  of  existing  systems. 
There  are  several  classifications  of  metrics  [Basil IJ.  Metrics 
can  be  divided  Into  product  metrics  (number  of  decisions  and 
interfaces)  and  process  metrics  (based  on  time  of  development, 
number  of  errors).  A  second  dichotomy  is  quality  metrics  (which 
would  evaluate  the  product  as  good  or  bad  relative  to  a  specific 
model)  vs.  Invariant  metrics  (which  are  independent  of  environment 
and  product  except  for  the  effects  of  size  on  cost). 

A  third  classification  would  distinguish  between  a  priori  metrics 
and  a  posterior  metrics.  An  a  priori  metric  Is  used  to  estimate 
and  evaluate  the  product  being  designed.  An  a  posterior  metric  Is 
a  measure  of  the  existing  product  after  the  design  Is  complete. 

It  Is  suggested  that  a  single  metric  Is  not  sufficient  for  large 
software  projects  and  hence  a  spectrum  of  metrics  should  be  used 
[PAM  85].  Metrics  to  be  used  should  be  selected  based  on  the 
objectives  of  a  particular  project.  There  Is  a  need  for 
experimental  validation  of  metrics,  but  little  validation  can  be 
expected  because  of  the  difficulty  in  conducting  controlled 
experiments  In  this  area. 

RESEARCH  ISSUES: 

Normally  a  designer  would  like  to  use  several  representation 
schemes  for  the  design.  The  various  representations  should  be 
checked  for  consistency,  I.e.  whether  they  could  represent  the 


same  system.  Tools  should  be  developed  that  provide  graphic  and 
textual  representations  based  on  several  models,  and  keep  the 
different  representations  consistent. 

The  human  expert  needs  tools  to  check  for  the  consistency  of  the 
requirements.  Tools  to  be  developed  include  browsers  for  browsing 
requirements  and  consistency-checking  theorem  provers  that  analyze 
the  system  being  designed. 

Prototypes  are  useful  for  requirement  evaluation.  We  would  like 
to  explore  how  the  predictive  approach  (metrics)  can  help  rapid 
prototyping  by  reducing  the  detail  and  amount  of  Implementation. 

There  Is  a  need  to  define  a  set  of  metrics  to  estimate  design 
understandab i I ity,  modifiability  and  complexity  from  the 
requirements.  A  typical  validation  process  can  be  used  [Boehm 
78],  [McCall  77].  The  selection  of  metrics  Is  subjective  and 
based  on  experience.  Then  one  clusters  the  metrics  by  factor 
analysis.  This  is  followed  by  collection  of  data  and  regression 
analysis  (the  "well  cycle"  involves:  building  models,  taking 
measurement,  and  performing  analysis  to  validate  the  models.)  A 
formal  model  of  requirements  evaluation  would  make  things  more 
concrete  by  providing  criteria  for  goodness  of  the  requirements. 
Metrics  can  be  defined  on  the  basis  of  that  model.  The  metric 
scoring  method  is  an  important  issue,  and  the  method  should  be 
defined  properly  and  then  automated. 

Test  cases  can  be  generated  from  the  requirements.  This  can  be 
achieved  through  transformation  or  via  expert  system  guidance. 
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SOLUTION  APPROACH  (2):  CRITIQUER  VIA  KNOWLEDGE-BASE/DOMAIN  KNOWLEDGE 


The  purpose  of  a  crltiquer  Is  two-fold.  (1)  It  Is  a  flexible  way 
to  measure  the  quality  of  requirements  expressions 
(understandab i I Ity,  clarity,  consistency)  and  requirements 
functionality  (modifiability,  evaluation  and  comparison  of 
alternatives,  complexity  and  feasibility).  (2)  The  critiquer  can 
be  used  to  automate  the  measurement,  analysis,  and  interpretation 
of  other  metrics.  This  is  a  fast  and  reliable  way  to  deal  with 
the  classical  metrics. 
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A  characterization  of  the  kinds  of  errors  that  can  be  present  In 
the  requ ! rements  would  Include:  errors  of 
Incons Istency/ Incompat lb  1 1  I ty,  errors  of  quality 
(understandab 1 1  tty,  over-constralntment,  redundancy,  risks  to 
cost/schedule  etc),  and  errors  of  Incompleteness  [Bell  763. 

Consistency  of  the  requirements  can  be  checked  with  the  assistance 
of  a  theorem-proving  tool.  By  adding  enough  knowledge  about  the 
domain  we  can  make  the  theorem  prover  a  better  assistant.  Qual ity 
can  be  measured  by  defining  appropriate  metrics  on  the  software. 
Still,  It  Is  Impossible  for  a  person  to  measure  these  properties 
for  a  very  large  software  system.  Similarly  the  Interpretation  of 
a  spectrum  of  metrics’  values  will  be  boring  and  difficult  for  a 
person. 

BACKGROUND: 

Rule-based  Expert  Systems  Is  an  established  technology  now.  This 
technology  has  provided  techniques  for  dealing  with  inherently 
ill-defined,  difficult,  large  and  complex  problems. 

Traditionally,  it  has  taken  a  long  time  for  one  to  gain  enough 
experience  to  become  an  expert  in  these  areas.  Knowledge  in  these 
areas  tends  to  be  inexact,  evolving,  and  difficult  to  formalize. 
The  rule-based  programming  paradigm  scores  will  over  the 
traditional  programming  paradigm  (e.g.  Lisp-based  systems  vs. 
traditional  Pascal  environments). 

The  power  of  the  rule-based  paradigm  comes  from  separation  of 
knowledge  and  reasoning.  This  makes  It  easier  to  add  knowledge 
about  the  domain  Incrementally.  It  also  facilitates  quick 
experimentation  and  modification  of  such  rule-based  systems. 

Rules  make  systems  such  as  the  crltiquer  adaptable  and  tailorable 
to  any  project.  Different  models  of  software  development 
(Waterfall,  Spiral  with  rapid  prototyping)  can  be  used  just  by 
changing  the  set  of  rules.  Rules  are  also  helpful  in  automating 
certain  tasks,  as  they  become  well  understood.  Rules  can  help 
maintain  standards  for  uniformity  and  quality  [Ram  85j. 

Research  in  Artificial  Intelligence  has  provided  several  knowledge 
representation  schemes  and  an  Inference  engine  technology  to  use 
that  knowledge.  There  are  tools  available  for  knowledge 
acquisition  (i.e.  the  transfer  of  problem-solving  expertise  from 
an  expert  to  the  program). 

RESEARCH  ISSUES: 

The  first  solution  approach  recommends  that  a  theorem  prover  be 
used  to  assist  designers  In  checking  for  consistency  of  the 
requirements.  Simple  tools  of  this  kind  can  help  locate 
requirements  which  potentially  give  rise  to  conflicts.  When  added 
to  the  knowledge  acquisition  system,  the  resulting  system  would 
learn  about  the  system  being  designed  as  well  as  assist  In 
defection  of  inconsistencies.  It  could  generate  test  cases  for 
checking  the  completeness  of  the  final  design.  The  knowledge 


acquisition  capability  helps  by  shifting  different  kinds  of 
checking  to  the  machine,  as  they  become  routine  and  better 
understood. 

It  is  widely  recognized  that  software  development  is  a  knowledge- 
intensive  process.  The  seemingly  Inherent  shortcomings  of  current 
approaches  to  software  development,  demonstrate  our  limited 
understanding  of  the  process  and  product  involved  [Arrango  85]. 
There  is  a  need  to  explore  the  nature  of  these  processes  and 
develop  explicit  representations,  so  that  we  can  reason  about 
them. 

One  should  separate  software  engineering  knowledge  from  domain 
knowledge  because  their  application  Is  relatively  orthogonal.  One 
should  also  create  good  representation  and  manipulation  schemes 
for  these. 

There  should  be  some  experimental  work  to  develop  a  set  of  rules 
to  criticize  the  properties  of  the  requirements  and  give  feedback 
to  the  user.  To  manipulate  the  metrics  one  has  to  design  suitable 
rules  for  measurement  and  Interpretation  of  data.  The  rules  will 
need  experimental  validation.  The  development  of  criteria  of  the 
goodness  of  requirements  is  also  needed.  These  developments  are 
complementary  to  the  modeling  of  software  engineering  knowledge. 

REFERENCES: 

1.  Arrango  G.,  Freeman  P.,  "Modeling  knowledge  for  software 
development"  Third  International  Workshop  on  Software 
Specification  and  Design  1985,  IEEE  Computer  Society. 

2.  Bell  T.E.,  Thayer  T.A.,  "Software  Requirements:  Are  they 
really  a  Problem",  Proceeding  2nd  International  Conference 
on  Software  Engineering  1976  IEEE. 

3.  Buchanan  B.G.,  Feigenbaum  E.A.,  "DENDRAL  and  Meta-DENDRAL", 
Artificial  Intelligence  11:1  (1978)  pp.5-24. 

4.  Clark  K.L.,  McCabe  F.G.,  "PROLOG:  a  language  for 
implementing  Expert  Systems",  Machine  Intelligence. 

5.  C.L.  Foggy,  "The  OPS-5  User’s  Manual"  Technical  Report, 
Carnegie  Mellon  University  1980. 

6.  Greenspan  S.  "Requirement  Modeling:  A  Knowledge 
Representation  Approach  to  Software  Requirements  Definition" 
CSRG,  U.  of  Toronto,  Tech.  Rep.  CSRG-155,  March  1984. 

7.  Leung  C.H.C.,  Choo  Q.H.  "A  Knowledge-base  for  affective 
Software  Specification  and  Maintenance",  Third  International 
Workshop  on  Software  Specification  and  Design  1985,  IEEE 
Computer  Society. 


8.  Dana  S.  Nau,  "Expert  Computer  Systems",  Computer  Feb.  1983, 
pp.  63-85. 

9.  Ramamoorthy  C.V.,  Garg  V.  and  Aggarwal  R.  "Environment 
Modeling  and  Activity  Management  In  GENESIS",  2nd  conference 
on  software  tools,  techniques  and  development  1985. 

COST: 

(1)  Metric  Guided  Methodology  3  person  yrs  .45  million 

-  and  metrics,  and  tools 

(2)  Knowledge-base  manipulation  3  person  yrs  .45  million 
of  design  knowledge 

(3)  Correlate  different  design  1  person  yrs  .15  million 
representat Ions 

(4)  Predictive  metrics  and  1  person  yrs  . 1 5  ml  I  I  ion 

prototyp i ng 

DELIVERABLE  PRODUCT: 

1.  Expert  System  Tools  for  knowledge  acquisition  and 
manipulation  of  knowledge  about  the  system  being  designed, 
for  locating  poTentlally  conflicting  requirements,  for 
critiquing  the  understandab 1 1 i ty ,  modifiability  and 
feasibility  of  requ I rements. 

2.  Metrics  for  measuring  the  understandab i I ity,  modifiability, 
feasibility  of  requ Irements,  and  complexity  of  later  stages 
of  design.  Tools  for  measuring  the  metrics  and  interpreting 
them. 

3.  Tools  to  support  several  different  representation  schemes 
for  the  same  design.  These  would  maintain  version  and 
configuration  Information  and  try  to  keep  different 
representation  for  the  same  design  consistent. 

4.  Methodology  to  combine  predictive  metrics  with  prototyping 
for  rapid  prototyping. 

RISK: 

The  tools  would  assist  a  designer.  Total  automation  is  not  being 
attempted  since  that  does  not  seem  feasible.  There  should  be  an 
attempt  to  make  the  tools  general  yet  tallorable  to  a  particular 
design  environment.  This  added  level  of  abstraction  Increases  the 
risk. 

AUTHOR  NAME : 

C.V.  Ramamoorthy 
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E.1.12  Testbed  Effectiveness 


RET  Research  Issue.  Evolutionary  Track. 


PROBLEM  DESCRIPTION: 

New  tools  and  techniques  for  requirements  engineering  will  be 
developed  because  of  the  early  leverage  they  bring  to  requirements 
problems.  To  ensure  that  the  Air  Force  employs  the  best  tools  and 
techniques,  their  effectiveness  In  producing  correct  requirements 
must  be  determined.  This  Is  a  key  purpose  of  the  requirements 
engineering  testbed. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Demonstrating  that  use  of  a  tool  leads  to  Improved  requirements 
and  better  productivity  will  assist  Its  adoption  by  the  Air  Force 
community.  Use  of  proven  tc-ols  will  lead  to  better  requ  i  rements. 

SOLUTION  APPROACH; 

The  following  things  will  be  measured  for  the  indicated 
characteristics: 

(1)  all  tools:  amount  of  user  time/effort  and  testbed  resources 
spent,  number  of  requirements  errors  caught, 

(2)  prototyping  (building  prototypes  to  determine  how  best  to 
adjust  requirements  to  cut  cost/risk,  and  Improve  schedule): 
estimated  savings  In  cost,  risk,  schedule, 

(3)  the  entire  requirements  engineering  process:  cumulative 
effort  and  testbed  resources,  number  of  errors  caught  (not 
cumulative,  but  before/after  comparison),  estimated  savings  In 
cost,  risk,  schedule,  and 

(4)  sensitivity  analysis:  same  as  3. 

Measurements  of  the  Improvement  In  requirements  In  3  and  4  will  be 
proportionately  allocated  back  to  utilized  tools  according  to  user 
effort  spent.  Before/after  comparison  of  requirements  will  be  a 
largely  manual  effort  assisted  by  general  documentation/versioning 
too  I s . 

5-YEAR  OBJECTIVES: 
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Testbed  instrumented  tor  effort  and  resource  measurements. 
Prototyping  providing  basis  to  cost,  risk,  and  schedule  estimates. 


10-YEAR  OBJECTIVES: 


Cost,  risk,  schedule  estimation  capability. 
User-preference  measurement  of  tools  at  end  of  process. 


It  will  be  hard  to  evaluate  the  effectiveness  of  individual  tools; 
easier  to  evaluate  the  effectiveness  of  the  entire  process.  5- 
year  and  10-year  objectives:  significant  risk  in  research 
strategies  because  measurements  can  not  reflect  subtle  tool 
Interactions  with  the  Testbed  user. 


COST: 


5-year  objectives:  2  man  years.  10-year  objectives:  3  man  years 
for  adopting  estimation  capabilities  developed  under  other  efforts 
(see  "Contingencies"),  1  man  year  to  provide  user-preference 
measurement. 


CONTINGENCIES: 


Dependent  on:  (1)  Testbed  Integration  effort:  instrumentation  of 
testbed  requires  tracking  all  testbed  activities.  (2) 
"Requirements  evaluation"  effort  and  "Estimation  of  cost,  risk, 
tine  in  system  development;  Performance  and  Execution  costs 
Analysis"  effort:  measuring  the  quality  of  the  requirements 
before/after  tool  or  technique  use. 


CONCLUSION: 


Testbed  effectiveness  is  mostly  a  development  issue,  and  might  be 
incorporated  in  the  "Testbed  integration"  effort.  It  is  not  clear 
whether  research  strategies  such  as  estimators  will  be  effective. 
Most  of  the  relevant  research  falls  under  the  two  research  Issues 
referenced  under  "Contingencies". 


AUTHOR  NAME: 


Stephen  Sherman,  Michael  Konrad. 
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ISSUE  NAME:  Testbed  Effectiveness 


OBJECTIVE: 

Provide  capabilities  to: 

(t)  Measure  user  time/effort  and  testbed  resources  spent. 

(2)  Perform  cost,  risk,  schedule  estimation. 

(3)  Compare  requirements  quality  before/after  application  of 
a  tool  or  techn Ique. 


SCOPE: 


Technical  areas  to  be  addressed:  Testbed  Instrumentation.  Cost, 
risk,  schedule  estimation. 

TECHNICAL  APPROACH; 

The  technical  approach  is  more  development  than  research. 

Objective  1:  Instrument  Testbed  project  tracking  functionality  to 
note  for  each  project  each  usage  of  a  tool,  collecting  effort  data 
and  before/after  requirements  versions. 

Objective  2:  Utilize  product  metrics  developed  under  the  two 
research  efforts  referenced  under  "Background"  for  measurement  of 
requirements  at  key  times  in  a  project,  for  determination  of  tool 
and  technique  effectiveness.  Adopt  as  appropriate  to  measurement 
of  prototypes. 

Objective  3:  Ensure  functionality  exists  to  highlight  differences 
between  two  versions  of  requirements,  one  ancestral  to  the  other. 

BACKGROUND: 

The  issue  is  unique.  There  are  no  similar  efforts.  References  on 
estimation  will  be  found  In  the  background  descriptions  of  two 
other  research  issues:  "Requirements  evaluation"  and  "Estimation 
of  cost,  risk,  time  In  system  development;  Performance  and 
Execution  costs  Analysis". 

DURATION: 


Testbed  Instrumentation 
Adopt  estimation  capabilities 
User-preference  measurement  support 


24  months 
36  months 
12  months. 


Testbed  instrumentation 
Adopt  estimation  capabilities 

(assuming  not  covered  by  another  effort) 
User-preference  measurement  support 

DELIVERABLES: 

PRODUCT:  DATE  (months  after  start) 

Testbed  instrumentation: 

Design  (project  tracking) 

Code 

Adopt  estimation  capabilities 

Identification  of  metrics  to  be  employed 
(report) 

Estimating  techniques  on  prototypes  (report) 
Metrics  for  requirements  descriptions  (code) 
Metrics  for  prototype  descriptions  (code) 
User-preference  measurement  support 

Report  on  what  evaluations  Testbed  user 
should  provide  on  project  completion 


.3  million 
.45  mil  I  ion 

.15  mill  ion. 


12  months 
24  months 


12  months 
18  months 
24  months 
36  months 


12  months 


*  Windowing  support  capabilities  allowing  the  user  to 
easily  manipulate  multiple  concurrent  tasks  and  manage 
multiple  views  of  Independent  data  bases/flies. 

*  User-Input  dialog  support  capabilities  for  uniformly 
communicating  with  the  end  user  In  a  consistent  and 
friendly  manner. 

*  Clipboard  support  capabilities  for  communicating 
arbitrary  textual  and  graphical  Information  between 
various  tool s/appl Icatlons.  (This  capability  should 
also  allow  and  support  tool  developer's  own 
definitions  of  complex  data  (self-defining  data 
structures  which  can  be  passed  between  cooperative 
tools  or  tool  fragments)  -  for  example  one  might  wish 
to  pass  a  tree  or  symbol  table  of  some  sort  between 
tool s. ) 

5  YEAR  OBJECTIVES: 

Clearly  establish  user  Interface  models  and  standards  to  be 
applied  to  all  prototype  and  operational  tool  development 
activities. 

10  YEAR  OBJECTIVES: 

Modify  and  maintain  user  interface  models  and  standards  in  an 
evolutionary  manner  as  appropriate. 


The  only  real  risk  associated  with  this  activity  Is  the  risk  of 
NOT  doing  it.  If  this  Is  Ignored  or  not  attended  to,  there  Is  a 
real  high  probability  of  much  of  the  research  effort  going  for 
naught. 


COST: 


There  must  be  an  Initial  effort  aimed  at  specifying  a  set  of 
candidate  standards  and  conventions  to  be  observed  In  the 
development  of  tool  interfaces.  This  Is  probably  on  the  order  of 
a  1/2  to  1  person  year  activity.  This  Initial  effort  will  define 
a  set  of  run  time  support  libraries  (or  Packages)  that  should  be 
used  by  subsequent  tool  builders. 

The  cost  of  the  run  time  support  capabilities  will  depend  upon 
whether  existing  capabilities  are  available  or  new  ones  must  be 
created.  (For  example,  Windowing  and  graphics  run  time  support 
will  clearly  be  required  -  whether  suitable  capabilities  are 
available  or  must  be  built  will  have  to  be  determined.) 

Subsequent  costs  should  really  be  transparent  to  the  main  research 
activities  and  Included  In  all  tool  development  activities  as 
standard  fare. 


ISSUE  NAME:  User  Interface 


OBJECTIVE: 


(1)  Develop  standards  and  conventions  to  be  observed  In  the 
development  of  tool  Interfaces  and  the  use  of  run-time  support 
packages. 

(2)  Check  compliance  with  standards  by  all  tool  contractors. 

(3)  Modify  standards  In  an  evolutionary  manner. 

SCOPE: 

Broad  standards  are  needed  early.  With  time,  selected  standards 
will  be  revised  and  refined.  Comp  I  I ance  check i ng  should  be 
thorough. 

TECHNICAL  APPROACH; 

Objective  1  is  already  well  addressed  in  the  "Solution  approach" 
and  "Cost"  sections.  Objective  2  might  be  accomplished  through  a 
Quality  Assurance  function  (perhaps  associated  with  testbed 
administration)  at  RADC'.  Objective  3  might  be  accomplished 
through  periodic  reviews  of  standards  for  possible  modification. 

An  alternative  approach  to  satisfying  objectives  2  and  3  is  to 
appoint  an  Independent  authority  who  Is  responsible  for  both 
checking  compliance  and  maintaining  and  evolving  standards.  This 
ts  the  approach  taken  below. 

BACKGROUND: 

Interactive  user  Interfaces  stress  current  underlying  system 
facilities  for  input/output  CO.  However,  there  are  notable 
trends  as  evidenced  recently  In:  (1)  "Macintosh  Style  Manual" 
standards  and  (2)  Windows  tool  interface  and  run-time  support 
standards. 

REFERENCES: 

CO  M.  Shaw,  "An  Input-Output  Model  for  Interactive  Systems", 
CHI'86  Proceedings,  April  1986,  ACM. 

CURAT  I  ON: 

Tool  interface  standards  12  months 

Modify  standards  and  do  tool  check  24  months 

(possible  renewal  every  24  months) 


MW 


Too!  Interface  standards  .15  million 

(cost  assumes  standard  run-time  packages  do  not  have  to 
be  developed) . 

Modify  standards  and  do  tool  check  .15  million 

each  24  months 

DELIVERABLES: 

PROOUCT:  DATE  (months  after  start): 

Tool  Interface  standards  (3  reports:) 

User  characterizations  3  months 

User  Interface  model  for 

menus,  text,  graphics,  forms  6  months 

Standards  12  months 

(on  tool  Interface  and  run-time  support  packages). 

Modify  standarcs  and  do  tool  check: 

Report  on  too!  compliance  with  standards  every  new  tool 
Revised  standards  every  12  months 
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E.1.14 


Evolutionary  Track. 


PROBLEM  DESCRIPTION: 


Provide  data  storage  and  data  management  capabilities  to  support 
tight  Integration  of  RET  tools  via  share  data. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Sharing  of  data  would  occur  on  the  following  objects:  data 
descriptions,  scenario  data,  requirements  descriptions,  procedure 
descriptions,  domain  Information,  other  text.  The  value  of 
sharing  comes  in:  minimizing  reentry  of  common  data;  fast 
conveyance  of  outputs  from  one  tool  to  Inputs  of  other  tools; 
assurance  of  data  consistency  between  tools;  maximizing  the 
availability  of  data  for  each  tool;  and  minimizing  development  of 
data  interfacing  code. 

SOLUTION  APPROACH: 


Purchase  a  commercial  database  system  which: 

1)  efficiently  supports  the  data  objects  used  in  software 
design  activities;  and 

2)  provides  standard  levels  of  support  for  distributed  data 
access,  security,  reliability,  etc.,  for  large  volumes  of 
data.  While  no  such  DBMS  now  exists,  as  one  becomes 
available  it  should  be  incorporated  in  the  RET.  Also  needed 
will  be  the  definition  of  data  descriptions  for  standard 
data  objects  used  in  communication  between  tools.  The 
assumed  solution  approach  is  to  have  common  data  stored  in  a 
canonical  form  in  the  database  and  use  Input/output 
conversion  facilities  to  put  external  data  Into  the  format 
required  by  each  tool. 

5  YEAR  OBJECTIVES: 

Within  five  years,  a  fully  effective  DBMS  should  be  in  place,  and 

conversion  standards  and/or  techniques  for  most  tools  will  be  in 


10  YEAR  OBJECTIVES: 


Incorporate  further  advances  in  DBMS  technology  Into  the  RET,  as 
appropr I  ate. 
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The  chance  of  obtaining  a  good  DBMS  is  high,  but  not  certain 

COST: 

1)  Evaluate,  obtain,  and  Install  a  DBMS; 

2)  develop  common  data  object  descriptions;  and 

3)  Create  conversion  routines  for  RET  tools  In  existence. 
About  one  to  three  man-years. 

DEPENDENCIES: 

None  within  RET 
CONCLUSION: 

This  Is  a  critical  element  of  the  evolutionary  RET  track. 
AUTHOR  NAME: 


Terry  Welch 


Evolutionary  Track 


PROBLEM  DESCRIPTION: 

Facilities  are  needed  to  manage  complex  design  information  in  the 
requirements  development  environment:  data  selection,  report 
generation,  and  storage  of  relationships.  This  would  apply  to  all 
design  data:  requirements  text,  prototype  specifications,  data 
descriptions,  project  management  data,  etc.  Critical  facilities 
would  support  managing  structured  text  as  a  complex  cf  design 
objects,  and  support  viewing  of  data  structures  from  a  variety  of 
projections  (e.g.  selecting  a  subset  of  a  large  system  description 
and  reordering  that  slice  of  the  description  according  to  some  new 
criterion) . 

VALUE  IN  REQUIREMENTS  PROCESS: 

These  facilities  will  be  necessary  to 

1)  extract  particular  subsets  of  information  for  specific  jobs, 
such  as  preparing  reports  for  specialized  audiences; 

2)  analyze  prior  project  information  for  reuseab 1 1 Ity ;  and 

3)  trace  decisions  between  various  parts  of  the  requirements 
design  process. 

S GLUT  I  ON  APPROACH; 

Use  a  general-purpose  DBMS  which  is  efficient  in  the  storage  of 
design  objects,  and  which  provides  the  proper  facilities. 

5  YEAR  OBJECTIVES: 

Such  a  system  should  be  In  place  within  5  years. 

10  YEAR  OBJECTIVES: 

Incorporate  further  advances  in  DBMS  technology  into  the  RET,  as 
appropr i ate. 


RISK: 


Some  kind  of  system  can  always  be  found.  The  risk  is  that  full 
capabilities  for  dealing  with  interrelated  structures  will  not  be 
found. 


Evaluate,  select,  install,  customize  for  RET  needs:  1  man-year. 
DEPENDENCIES: 

This  DBMS  should  be  the  same  one  used  for  sharing  data  between  RET 
tools.  The  sharing  function  will  more  strongly  stress  the  data 
modeling  capabilities  of  the  DBMS. 

CONCLUSION: 

This  will  be  the  highest  pay-back,  lowest  cost  element  in  the  RET. 
It  must,  however,  await  the  development  of  a  proper  DBMS  for  data 
sharing,  so  that  the  database  will  be  populated  with  relevant 
project  information. 

AUTHOR  NAME: 


Terry  Welch 


ISSUE  NAME:  Database  -  Shared  Data  &  Data  Management  Facilities 
OBJECTIVE: 


Provide  facilities  which  aid  RET  users  and  RET  tools  In 

collecting,  accessing  and  managing  large  volumes  of  complex  data. 

Including 

1)  Viewing/Reporting  -  the  ability  to  extract  specified  subsets 
of  data  elements  and  present  those  subsets  of  data  in  a  way 
meaningful  within  application  areas,  often  employing 
graphical  presentation  means. 

2)  Editing  -  Directly  collecting  and  modifying  database 
information. 

3)  Classification  -  capture  of  information  from  specific 
applications,  and  organization  of  that  information  to  make 
it  meaningful  for  later  access. 

A)  Export/ I mport  -  formatting  data  for  use  by  external  tools. 

SCOPE: 


It  is  expected  that  this  effort  will  be  based  on  use  of  an 
existing  commercial  database  system  which  has  these  properties: 

1)  it  efficiently  stores  design  representations,  meaning 
complex  object  structures; 

2)  It  provides  conventional  viewing  and  report  generation 
f ac II  it les;  and 

3)  support  for  team  development  through  version  control  and 
distributed  access.  The  proposed  effort  for  the  RET  occurs 
In  defining  effective  application  specific  data  models  and 
classifications,  so  that  the  general  tools  of  DBMS  are 
available  through  the  RET  standard  user  Interface  and 
through  procedural  Interfaces. 

TECHNICAL  APPROACH: 

This  work  involves  conventional  systems  analysis  for  a  non- 
conventional  application,  namely  understanding  application  data 
types  and  information  flow,  and  mapping  those  onto  the  tools 
provided  by  the  DBMS.  A  critical  aspect  of  the  work  will  be  the 
choice  of  the  DBMS;  many  possible  candidates  will  be  poorly 
matched  to  this  job. 


*8 


BACKGROUND: 


Many  of  the  conceptual  Issues  being  addressed  In  this  effort  are 
also  being  studied  elsewhere: 

1)  the  Air  Force  directed  VHSIC  effort  on  Engineering 
Information  Systems  (EIS);  and 

2)  efforts  by  several  small  companies  to  build  object-centered 
design  database  systems.  The  concepts  Involved  in  the 
storage  and  access  of  design  information  should  mature 
significantly  over  the  next  couple  of  years. 


REFERENCES: 


Not  yet  publicly  available. 


DURATION: 


COST; 


Cnee  a  suitable  DBMS  is  available,  an  eighteen-month  effort  should 
suf f Ice. 


.45  million  assuming  a  3  man-year  effort. 


DELIVERABLES: 


Software  to  be  executed  on  the  RET  (Apollo  system  plus  licenses 
for  incorporated  DBMS  and  graphics  software  packages  as 
appropriate).  Reports  on  analysis  of  user  data  and  procedures 
should  be  available  In  six  months,  a  DBMS  selected  and  installed 
in  twelve  months,  and  application/tool  specific  interfaces 
provided  in  eighteen  months. 
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Evolutionary  Track. 


PROBLEM  DESCRIPTION: 


To  maximize  the  benefit  from  the  testbed,  all  developed  or 
acquired  tools  should  be  capable  of  working  together,  and  thus 
they  need  to  be  Integrated.  Also,  a  key  purpose  of  the  testbed  Is 
to  support  experiments  comparing  different  tools  and  techniques; 
this  requires  monitoring  tool  use  and  controlling  parameters. 

Such  Is  made  easier  In  an  integrated  testbed. 


VALUE  IN  REQUIREMENTS  PROCESS: 


Key  capabilities  of  different  tools  can  be  brought  to  bear  on  the 
same  requirements  problems,  leading  to  improved  requ Irements. 


SOLUTION  APPROACH; 


To  "bring"  the  tools  together:  modify  tools  to  utilize  the  common 
database  and  common  user  interface  developed  under  "Database"  and 
"User  Interface"  development  issues.  Instrument  the  testbed  for 
tool  tracking  and  measurement. 


To  make  effective  use  of  the  testbed,  a  "Testbed  usage 
methodology"  should  be  developed. 


5-YEAR  OBJECTIVES: 


An  Integrated  testbed: 

tools  have  a  common  user  Interface  and  database, 
testbed  activities  tracked  and  measured,  and  a 
testbed  usage  methodology. 


10-YEAR  OBJECTIVES: 


Incorporate  new  tools  and  techniques. 

Further  testbed  evolution  toward  data-or ientat ion  from  tool- 
oriented  Interface. 


One  risk  is  Integration  made  unnecessarily  tight  (wasted  effort). 
Determining  when  functionality  is  sufficiently  mature  for  data- 


directed  Invocation  is  tricky.  Current  RADC  contractors  need  to 
cooperatively  develop  their  tools  to  aid  later  integration. 

COST: 

5-year  objectives:  3.5  man  years.  10-year  objectives:  3  man 
years. 

CONTINGENCIES: 

This  development  issue  should  subsume  the  testbed  Instrumentation 
activity  discussed  in  "Testbed  effectiveness".  Providing  a  common 
user  Interface  and  database  for  the  tools  and  standards  is 
addressed  under  "User  Interface"  and  "Database"  development 
issues.  The  capability  for  data-d 1 rected  invocation  should  be 
covered  by  the  "Database"  issue. 

CONCLUSION: 

To  make  the  testbed  greater  than  the  sum  of  Individual  tools,  the 
Evolutionary  testbed  Integration  effort  must  be  funded.  The  case 
for  funding  long-term  objectives  is  less  clear. 

AUTHOR  NAME: 

Stephen  Sherman,  Michael  Konrad. 


RET  RAD  Effort 


ISSUE  NAME:  Evolutionary  Testbed  Integration 
OBJECTIVE: 

Develop  an  Integrated  testbed  featuring: 

(1)  a  common  database  and  user  Interface  for  tools, 

(2)  capability  for  evolution  toward  data-d Irected  Invocation, 

(3)  tracking  and  measurement  of  testbed  activities, 

(4)  a  testbed  usage  methodology,  and 

(5)  a  way  of  bringing  other  tools  In. 


SCOPE: 


Technical  areas  to  be  addressed:  Experiment  Methodologies. 
TECHNICAL  APPROACH; 

First  determine  the  minimal  degree  of  integration  necessary  to  get 
the  tool  interaction  and  tracking  desired.  Identify  what 
functionality  should  be  autcmat ical ly  invoked  by  changes  in  data, 
and  what  should  be  under  explicit  user  control.  Identify  what 
aspects  of  tool  use  are  to  be  tracked;  modified  tools  should 
automatically  record  details  of  each  usage.  In  short,  all  user 
and  tool  activities  will  be  registered  In  the  common  database, 
providing  a  basis  for  project  tracking  and  measurement. 

A  testbed  usage  methodology  should  guide  experimental  use  of  the 
testbed  facility.  Each  user's  project  (experiment)  should  include 
a  pre-testbed  phase  identifying  experiment  hypothesis  and  a  post¬ 
testbed  analysis  of  results. 

Future  contractors  should  develop  their  tools  to  directly  fit  In 
the  testbed  (use  common  database,  accessed  through  common 
i nterf ace) . 

A  subsequent  integration  contract  will  address  evolution  of  the 
testbed:  getting  new  functionality  to  work  with  the  old,  getting 
more  functionality  automatically  invoked,  and  improving  what  is 
tracked. 

BACKGROUND: 

The  issue  is  unique.  There  are  no  known  similar  efforts.  The 
references  cite  the  currently  planned  prototyping  tools  that  will 
first  be  Integrated. 
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REFERENCES: 


Dj  M.  Stephens  and  !<.  Whitehead,  "The  Analyst  -  A  Workstation  for 
Analysis  and  Design",  Proc.  8th  Int.  Conf.  Software  Eng., 
IEEE  Comp.  Soc.  Press,  1985. 

C23  R.  Yeh  and  M.  Konrad,  "VHLL  System  Prototyping  Tools",  a 
proposal  submitted  to  Rome  Air  Development  Center. 

C3]  P.  Daley,  "C^ I  Rapid  Prototype  Investigation",  Rome  Air 
Development  Center,  RADC-TR-85-216. 

DURATION: 


Integration  of  current  tools 

24 

months 

Testbed  usage  methodology 

12 

months 

Integration  of  future  tools 

36 

months 

COST: 


Integration  of  current  tools 

.45 

m  i  1 

1  ion 

Testbed  usage  methodology 

.07 

m  i  1 

1  ion 

Integration  of  future  tools 

.45 

m  i  1 

1  ion 

DELIVERABLES: 


PRODUCT: 


DATE  (months  after  start): 


Integration  of  current  tools 

Report  on  degree  of  integration  required 
Modified  tools  (design) 

Med i f i ed  tool s  (code) 

Testbed  usage  methodology  (report) 

Integration  of  futu.  e  tools 

Identify  functionality  to  be  incorporated 
Modify  tools  (design) 

Mod  I fy  tool s  (code) 


5 

months 

9 

months 

12 

months 

12 

months 

(report)  18 

months 

28 

months 

36 

months 

.  Formal  Language  Track 


PROBLEM  DESCRIPTION: 

Requirements  are  currently  informal.  Hence  computer  tools  can  do 
no  more  than  keep  track  of  the  requirements  statements  and  human 
claims  about  tracking  and  satisfaction  (i.e.,  electronic  note  pad 
and  record  keeping).  Requirements  are  thus  merely  guiding 
comments  for  human  consumption  and  the  requirements  process  Is 
supported  by  management  of  people  through  methodology. 

The  goal  is  the  replacement  of  this  informal  basis  by  a  formal 
treatment  of  requirements  and  automated  tool  support  for 
requirements  design  and  tracking  into  a  specification. 

VALUE  IN  REQUIREMENTS  PROCESS: 

Extremely  high.  Value  arises  from  earlier  detection  of 
inconsistent  or  unsat isf i ab I e  requirements,  better  trade-off 
analysis,  and  earlier  detection  of  requirements  unsatisfied  in  a 
specification,  particularly  ones  only  partially  (or  sometimes) 
satisfied. 

SOLUTION  APPROACH: 

Expand  existing  formal  specification  language  to  include  formal 
requirements  statements.  Share  a  common  domain  model  and  define 
requirements  as  predicates  against  behavior  of  specification. 
Formally  execute  specification  to  generate  behavior  against  which 
to  test  requirements  predicates.  Include  goals  as  requirements 
which  are  only  "desired."  Provide  support  for  multiple  levels  of 
abstraction  in  stating  requirements  and  specifications  and  mapping 
between  them.  Provide  support  for  evolving  requirements  statement 
on  basis  of  feedback  from  evaluation  tools. 

5  YEAR  OBJECTIVES: 

Common  formal  language  for  requirements,  specifications,  and  goals 
which  share  the  same  domain  and  behavior  models  and  methodologies 
for  dealing  with  these  formalisms.  Provide  tool  which  determines 
whether  requirements  are  satisfied  by  a  specification  with  respect 
to  a  particular  scenario. 

10  YEAR  OBJECTIVES: 

Expand  requirements  satisfaction  determination  tool  to  handle 
classes  of  scenarios  (via  symbolic  execution)  and  automatic 
determination  of  scenarios.  Support  multiple  levels  of 
abstraction  for  requirements  and  specifications  and  the  mappings 


between  them.  Manage  the  human  and  computing  resources  engaged  in 
the  evolution  of  requirements. 


RISK: 

Relatively  low  for  short  term  objectives  to  product  integrated 
formal  requirements  and  specification  language  and  to  formally 
execute  a  specification  against  a  scenario  to  generate  behavior, 
and  to  use  that  behavior  to  determine  whether  requirements  are 
satlsf led. 

Relatively  high  over  the  long  term  to  build  reasoning  and  analysis 
tools  which  can  handle  practical  size  requirements  statements  and 
provide  deep  and  comprehensive  feedback  to  aid  iterative  design  of 
requ I rements. 

COST :  8.7  million 

DEPENDENCIES: 

Requires  more  highly  trained  Requirements  Engineer  and  Mission 
User,  Fntire  approach  is  dependent  upon  integration  of 
requirements  into  Formal  Specification  language,  but  this  is 
relatively  low  risk.  Incremental  requirements  component  assumes 
successful  prior  completion  of  corresponding  incremental 
specification  effort  (separately  funded). 

CONCLUSION: 

Formal  language  approach  to  requirements  is  highly  recommended  as 
complement  to  evolutionary  track.  It  is  higher  risk  and  higher 
payoff  and  that  payoff  occurs  later  than  in  evolutionary  track. 

But  it  lays  the  foundation  for  earlier  and  more  reliable  detection 
of  requirements  problems  and  their  use  as  a  real  design  envelope. 

AUTHOR  NAME: 


Robert  Bal zer 
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1.  Formal  Language  Track 


OBJECTIVE: 


Integrate  requirements  Into  formal  specification  language,  sharing 
common  domain  and  behavior  models.  Resultant  language  must  be 
executab  I  e. 


SCOPE: 


Technical  areas  to  be  addressed:  Formal  semantics,  domain 

models,  database  schema,  executable  specifications,  formal 
specification  languages. 

Technical  areas  not  addressed:  Advances  in  formal  semantic 
theory,  new  formal  languages,  symbol ic  evaluation. 

TECHNICAL  APPROACH: 

Survey  existing  executable  specification  languages.  Choose  one 
with  an  explicit  domain  model  and  formal  model  of  behavior. 
Define  requirements  as  predicates  against  behavior  and  integrate 
into  language. 

RELEVANT  WORK: 


OBJ  (SRI),  Paisley  (AT&T),  Gist  ( I S I ) ,  Clear  (Edinburgh 
Un i vers ity ) . 


DURATION: 


24  months 


COST :  1  million 


DELIVERABLES: 


Formal  requirements  and  spec  language, 
shared  domain  model 


12  months 


Formal  requirements  and  spec  language, 
shared  behavior  model 


24  months 
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Formal  Language  Track 


OBJECTIVE: 


Execute  specification  against  scenario  to  generate  Its  behavior. 
Determine  whether  requirements  were  satisfied  by  this  behavior. 


SCOPE: 


Technical  areas  to  be  addressed:  Symbolic  evaluation,  formal 

semantics,  temporal  logic,  executable  specification. 

Technical  areas  not  addressed:  Advances  In  formal  semantic 
theory  or  logic. 

TECHNICAL  APPROACH: 

Use  symbolic  evaluator  to  generate  the  temporal  behavior  of  an 
executable  specification  on  a  scenario.  Extend  symbolic  evaluator 
with  additional  phase  that  determines  whether  requirements  are 
satisfied  by  this  behavior.  If  not.  Inform  user  which 
requirements  were  violated  by  what  portions  of  the  behavior  and 
how  this  resulted  from  the  specification. 

RELEVANT  WORK: 

Gist  symbolic  evaluator  and  behavior  explainer  (ISI),  ELI  symbolic 
evaluator  (Harvard),  Paisley  evaluator  (AT&T),  scenario 
spec i f ication. 

DURATION:  24  months 

COST:  1  million 

DELIVERABLES: 


Symbolic  evaluator  for  requirements 
and  spec  language 

Requirements  predicate  checker 

Requirements  violation  checker 


12  months 


21  months 


24  months 
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1.  Formal  Language  Track 


OBJECTIVE: 


Provide  guidance  for  Mission  Users  and  Requirements  Engineers  In 
creating  formal  requirements  and  using  facilities  of  RET  formal 
track. 


SCOPE: 

Technical  areas  to  be  addressed:  None. 

TECHNICAL  APPROACH: 

Extend  structured  specification  methodologies  to  requirements  and 
their  formal  expression.  Define  methodology  for  using 
requirements  checker  and  modifying  requirements  statement.  Define 
methodology  for  stating  goals  and  determining  which  should  become 
requirements.  Design  experiments  to  test  and  validate  proposed 
methodologies. 

RELEVANT  WORK: 

Structured  Design,  CORE,  metrics. 

DURATION:  36  months 

COST :  1  million 

DELIVERABLES: 


Structured  requirements  synthesis 
methodology 


12  months 


Guidelines  for  use  of  requirements 
checker 


24  months 


Methodology  for  using  goals 


30  months 


Requirements  methodology  manual 


36  months 


.  Formal  Language  Track 


OBJECTIVE: 

Provide  feedback  to  user  about  degree  of  coverage  of  goal 
satisfaction  and  degree  to  which  satisfaction  is  obtained. 
Provide  basis  for  trade-off  analysis. 


SCOPE: 


Technical  areas  to  be  addressed:  Symbolic  evaluation,  test 

coverage  analysis,  decision  support  systems. 

TECHNICAL  APPROACH: 

Extend  requirements  checker  to  keep  track  of  cases  for  which  goals 
were  not  satisfied.  Characterize  degree  of  coverage  from  model  of 
search  space.  Define  measures  of  satisfiability  and  algebra  for 
combining  values. 

RELEVANT  WORK: 

DURATION:  24  months 

COST :  1  million 

DELIVERABLES: 

Tracker  of  unsatisfied  goals 

Model  cf  search  space  and  estimator  of 
degree  of  goal  coverage 


6  months 

12  months 


Measures  of  satisfiability  and  algebra 
for  their  combination 


24  months 
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RET  R&D  Effort.  Formal  Language  Track 


OBJECTIVE: 

Formalize  tracking  of  requirements  through  multiple  levels  of 
abstraction. 

Requirements  are  predicate  against  behavior.  Hence,  refinement 
results  In  one  or  more  requirements  which  together  should  Imply 
the  original  requirement.  The  methodology  for  formal  requirements 
synthesis  will  provide  guidance  for  when  and  how  such  refinements 
occur.  This  effort  will  ensure  the  validity  of  each  such 
refinement,  keep  track  of  which  higher  level  requlrement(s)  are 
(partially)  satisfied  by  a  lower  level  one,  and  identify  what 
extra  assumptions/commitments  resulted  from  the  refinement. 


l 


Technical  areas  to  be  addressed:  Formal  models,  abstract  cata 
types. 

Technical  areas  not  addressed:  Automatic  classification, 

knowledge  representation  theory. 

TECHNICAL  APPROACH: 

Provide  formal  basis  for  defining  abstract  models  and  for  relating 
activity  In  one  to  the  corresponding  activity  In  the  other. 

Provide  tool  which  verifies  that  a  requirement  in  one  model  is 
ensured  by  a  set  of  requirements  In  another  model.  Provide  a  tool 
which  finds  that  set,  if  it  exists  (this  Is  a  formal  requirements 
tracker) . 

RELEVANT  WORK: 

Automatic  classification,  knowledge  representation,  theorem 
prov i ng. 

DURATION:  36  months 

COST:  1.5  mill  Ion 

DELIVERABLES: 

Abstract  Nodel  Definition  facility  12  months 

Verifier  that  a  requirement  In  one 
model  is  covered  by  a  set  of  refinements 

in  another  model  24  months 
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Identification  of  extra  assumptions/commitments 
resulting  from  a  refinement  30  months 

Formal  Requirements  Tracker 


36  months 


.  Formal  Language  Track 


OBJECTIVE: 

Scenarios  are  crucial  to  testing  requirements  and  the  systems  that 
(purport  to)  Implement  them.  Whereas  requirements  and 
specifications  are  general  statements,  which  are,  respectively, 
predicates  against  behavior  and  generators  of  that  behavior, 
scenarios  define  behavior  for  specific  cases.  As  such,  they  can 
be  used  as  test  cases  to  verify  that  both  requirements  and 
specifications  Include  the  desired  specific  behavior. 

However,  to  be  used  in  this  manner,  a  scenario  must  be  both 
complete  and  detailed.  It  must  not  only  define  the  desired 
behavior,  but  also  all  the  inputs  and  controls  needed  to  ensure 
that  this  behavior  will  result.  Furthermore,  some  comparison 
mechanism  must  exist  to  determine  whether  the  behavior  generated 
by  the  specification,  which  is  more  complete  and  detailed  than  the 
subset  specified  in  the  scenario,  is  a  valid  instantiation  of  that 
desired  behavior. 

These  requirements  currently  make  scenario  generation  a  demanding, 
labor  intensive,  error-prone,  and  time-consuming  task.  However, 
with  formal  requirements  and  specifications,  much  of  this  effort 
can  be  automated. 

The  goal  of  this  effort  is  to  generate  a  complete  scenario  from  a 
mission  user’s  outline  of  desired  behavior. 


SCOPE: 


Technical  areas  to  be  addressed:  Test  case  generation,  symbolic 

evaluation. 

TECHNICAL  APPROACH: 

Reverse  the  flow  of  reasoning  In  symbolic  evaluation  so  that,  for 
a  given  specification,  partially  specified  behavior  ( I . e. , 
symbol ic  output)  can  be  used  to  characterize  the  Inputs  necessary 
to  generate  that  behavior.  Such  a  ’’backward  evaluation”  would 
build  up  parameter Ized  expressions  and  constraints  which  defined 
the  sets  of  Inputs  that  would  generate  the  behavior.  Consistently 
instantiating  these  expressions  (without  violating  the 
constraint;,)  would  provide  a  specific  scenario  for  the  desired 
behavior. 

Combinatorial  explosion  In  the  search  space  will  nececs'+ate  the 
use  of  heuristics  to  control  both  the  ’’backward  evaluation”  and 
the  parameter  instantiation  processes.  These  heuristics  should 


combine  knowledge  of  generating  test  cases  and  of  normal 
situations  and  behaviors  In  the  target  domain. 

Capabilities  produced  for  mapping  and  matching  multiple  levels  of 
abstraction  would  be  used  to  determine  whether  a  specific  behavior 
generated  from  a  specification  or  prototype  satisfied  the  mission 
user’s  statement  of  desired  behavior. 

RELEVANT  WORK: 

Symbolic  Evaluation,  Test  case  generation,  multiple  levels  of 
abstraction,  heuristic  search,  diagnostic  systems. 

DURATION:  24  months 

COST :  1  million 

DELIVERABLES: 

Scenarios  generation  from  Mission 

User’s  behavior  spec  18  months 

Determination  of  satisfaction  of 
actual  behavior  against  Mission 

User's  behavior  spec  24  months 


.  Formal  Language  Track 


OBJECTIVE: 

Provide  language-based  support  for  evolving  requirements  rather 
than  creating  them  from  scratch  all  at  once. 

Requirements  evolve  as  Mission  Users,  Acquisition  Engineers,  and 
Developers  think  more  carefully  about  the  system  to  be  built  and 
get  feedback.  Insight,  and  experience,  from  analysis  tools  and/or 
prototypes.  Yet,  formal  languages  capture  none  of  this  time 
history.  Each  stage  Is  a  stand-alone  complete  description  related 
to  the  others  only  by  a  version  number.  Each  stage  must  be 
produced  by  low-level  text  edits  of  the  previous  stage  which 
effect  the  Intended  modification  but  keep  that  modification 
imp  I  ic  it. 

This  effort  directly  supports  evolution  by  providing  explicit 
language  constructs  for  common  modifications.  These  constructs 
increase  comprehension  for  both  humans  and  tools  by  focusing 
attention  on  the  changes  and  by  providing  an  incremental,  staged 
basis  for  understanding. 

SCOPE: 

Technical  areas  to  be  addressed:  Formal  languages, 

transformat  ions. 


TECHNICAL  APPROACH: 

Build  upon  prior  work  in  incremental  specification  languages  for 
base  of  modification  constructs,  human  reading  tools,  and  tool 
focusing  capabilities.  Add  special  support  for  refining 
predicates  (requirements),  strengthening  and  weakening  them,  and 
revising  them. 

RELEVANT  WORK: 

Monotonic  logics  and  languages,  cognitive  models,  transformation, 
multi-level  models. 

DURATION:  36  months 

COST :  1 .5  mi  I  I  ion 

DELIVERABLES: 

Incremental  requirements  refinement 

constructs  6  months 
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Incremental  requirements  strengthening 
and  weakening  constructs 


12  months 


Incremental  requirements  revision 

constructs  18  months 

Integrate  incremental  I anguage  constructs 
with  multi-level  requirements  tracking 

tool  36  months 
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E.2.8  Man; 


1.  Formal  Language  Track 


OBJECTIVE: 


Manage  the  human  and  computing  resources  needed  to  create  an 
Initial  set  of  requirements,  analyze  them,  gather  feedback  from 
prototypes  (specifications),  determine  whether  they  are  satisfied 
by  the  prototype’s  behavior.  Iteratively  revise  and  refine  them, 
and  track  them  throughout  this  process. 


SCOPE: 


Technical  areas  to  be  addressed:  Task  representation,  agenda 

management. 


TECHNICAL  APPROACH; 


Provide  formal  support  for  activities  informally  defined  by 
previous  methodology  guidelines  effort.  Formally  represent  the 
tasks  required.  Identify  their  preconditions,  resources,  and 
results.  Construct  manager  which  understands  these  dependencies 
and  guides  users  in  task  selection. 


RELEVANT  WORK; 


KBSA  Activity  Coordinator,  CAD 


DURATION;  24  months 


COST :  .75  mi  I  I  ion 


DELIVERABLES: 


Formal  representation  of  RET  tasks 


6  months 


Task  manager  for  single  user 


12  months 


Task  and  tool  manager  for  single  user 


18  months 


Task  and  tool  coordinator  for  multi-user 
requ I rements  effort 


24  months 


•sy, 
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APPENDIX  F:  PHILOSOPHICAL  CONSIDERATIONS 


Determining  whether  requirements  are  complete  In  the  general  case 
Is  Impossible.  However  by  carefully  limiting  oneself  to  a  finite 
number  of  well-defined  properties,  and  defining  completeness  as 
satisfaction  of  these  properties,  It  Is  possible  to  determine 
whether  the  requirements  are  complete.  An  example  set  of 
properties  (from  CO  Is: 

*  No  TBDs, 

*  No  nonexistent  references, 

*  No  missing  specification  Items, 

*  No  missing  functions, 

*  No  missing  products. 

The  problem  In  the  general  case  Is  that  we  don't  know  all  the 
properties  because  we  lack  good  meta-models  covering  all  that 
which  we  are  trying  to  specify. 

References: 

Cl 3  B.  Boehm,  "Verifying  and  Validating  Software  Requirements  and 
Design  Specifications",  IEEE  Software,  January  1984. 


The  mechanisms  used  to  express  requirements  affect  the  choice  of  a 
solution.  There  Is  no  resolution  to  this  problem. 

Simply  stating  a  problem  necessarily  bounds  the  corresponding 
solution  space.  Formal  expression  exacerbates  the  bounding  (there 
Is  less  ambiguity).  Even  the  formalization  of  an  application 
domain  which  Is  subsequently  referenced  In  several  requirements 
imposes  bounds  on  the  corresponding  solution  spaces. 

Where  does  the  bounding  come  from?  Forma!  descriptions  are 
necessarily  a  mixture  of  organizational/structural  expressions 
(designs)  and  expressions  of  bounds.  While  the  descrlber's  goal 
Is  the  expression  of  the  latter  (the  bounds),  their  expression 
must  be  made  In  the  context  of  the  former  (the  designs).  The 
problem  Is  that  these  designs  themselves  produce  bounds,  and  these 
are  in  addition  to  those  bounds  the  descrlber  may  have  Intended  to 
exp  I icitly  state. 

The  conclusion  of  this  Is  that  requirements  and  solution 
architectures  contain  similar  kinds  of  descriptions  (designs  and 
bounds)  and  so  there  should  be  a  sharing  of  the  syntax  and 
semantics  used  In  expressing  them. 

Testbed  users  need  to  be  made  aware  that  the  form  of  expression 
they  employ  may  bound  the  solution  space  in  undesirable  ways. 
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MISSION 

of 

Rome  Air  Development  Center 

RAVC  plan*  and  <ixe.cu.tz-!>  research ,  development,  test 
and  selected  acquisition  pA.ogn.ams  In  support  ofi 
Command,  Contnol,  Communications  and  Intelligence 
( C 3 1 )  activities.  Technical  and  enqlneenlng 
suppont  within  aneas  ofi  competence  Is  provided  to 
ESV  Pnognam  06ilc.es  ( POs )  and  othen  ESV  elements 
to  pen.6on.rn  elective  acquisition  06  C^I  systems. 

The  aneas  06  technical  competence  Include 
communications ,  command  and  control,  battle 
management ,  In  fiormatlon  pKo cessing ,  surveillance 
sensors,  Intelligence  data  collection  and  handling, 
solid  state  sciences,  electromagnetics ,  and 
propagation,  and  electronic,  maintainability , 
and  compatibility . 
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